SteemWorld ~ Weekly Support ~ #23
[EN]
This post is intended to cover the server costs for SteemWorld.org and to enable me to continue my development in the future.
In case you missed the first part:
SteemWorld ~ Weekly Support ~ #1
[DE]
Dieser Post ist dafür gedacht die Serverkosten für SteemWorld.org zu decken und mir in Zukunft weitere Entwicklungen zu ermöglichen.
Falls du den ersten Teil verpasst haben solltest:
SteemWorld ~ Weekly Support ~ #1
Updates
I'm in the last steps of switching to the new server and my time is running out, so I will keep it short today. The plan was to start the new server directly with my coming SDS (Steem Data Service), but even if I made great progress in the past week, it is not ready to be used in production mode yet, therefore I need to setup my old PHP based web service and use it as a temporary solution there. Otherwise many features like mentions and the follower history wouldn't work in the coming week.
I think it's a better way to have a smooth transition, optimize SDS as much as possible to get the best out of it and finally release it, when it's really ready and stable enough to see the world. SDS includes never seen compression techniques, so that the blockchain can be stored in a super compact format. Trust me, we will need something like this when Steem goes mainstream and servers begin to run out of disk space.
Some of you may know the XZ compressor (based on LZMA), which on a level 9 (extreme mode, param: -e) setting compresses blocks to approximately 1/8 of the original size. With my SDS it is possible to compress them to a size that is ~ 0.63 times the size of the best XZ compression result. I hope that I will find the time in the coming week for an additional post to explain how I achieved that and to talk about what I'm really building here.
For most SteemWorld users that's all not really important to know, because it has to do with laying the groundwork for the future and coming SteemWorld features on the server side. I hope you don't mind, but a few 'boring' posts may still need to come until the more interesting things and the new SteemWorld features will see the light.
Coming Open Source Steem Data Service (SDS)
Some basic points I already can state here:
- Runs on Linux, Mac and Windows
- Can be run locally or as public web service
- Enables
each of usto analyze and work with the Steem blockchain data in a simple and very efficient way (SQL and custom queries can be used too) - Works as REST API:
//[domain or localhost]/sds/get_block/1
=> returns block 1 as JSON
/sds/get_blocks/1-1000
=> returns 1000 blocks (1 to 1000) with one request in second(s)
/sds/get_followers/steemchiller
=> returns my followers
... - Syncing is possible via available public instances (very fast) or regular Steem nodes
- Provides automatic downloading of already parsed data (no need to start from block 1)
- No need to setup any database services (all file-based -> no layers in between -> lightning fast)
- Offers configurable datasets (if you just need a DB with all transfers for example, you can run an instance with only the desired parser module 'transfers' enabled)
- JS developers can easily implement own parser modules (just add a custom script in the modules path and the service will call the
parseBlock,parseTransaction,parseOperationandparseVirtualOperationmethods within your script for each block) - Also 'virtual operations' can be parsed (or downloaded from one of my servers that will provide all parsed datasets in nearly realtime)
- Highly compressed data (some crazy stuff and a bit of magic involved here, more details on this soon)
- Interfaces to other database systems planned
- Completely free to use, will be released under MIT license
In the making
- Open Source Steem Data Service (SDS)
- Post Viewer & Editor
- Different views / no limit in the posts overview
- Details for coming rewards (number of posts/comments, maybe a pie chart)
- Improved coming curation rewards overview
- RC costs calculation
- SteemWorld -> Open Source?
Links
EN | What is SteemWorld?
-> Welcome to SteemWorld.org!
DE | Was ist SteemWorld?
-> Willkommen auf SteemWorld.org!
Thank you for supporting my work!
Positive thoughts create a positive world. It all begins with connecting our thoughts with emotions. What we focus on is what becomes powerful. We are the creators of our world.
Much love,
Weiter so 👍👏
Thanks for all your hard work @steemchiller, and particularly for the thorough and measured approach you take to keep up and release product improvements when they are ready and working properly! Having worked with many in the IT industry, that's a great approach to have... because many tend to work on a build, and then "just throw it out there and hope," after which a long stream of breakdowns and patches follow.
So thanks for the work you do, and the WAY you do it!
First, thank you for all the awesome work you do...
Second, I have a question if you don't mind. I'm trying to use steemworld to "create" an account (I have successfully "claimed" many thanks to your tools), but I keep getting an error message when using steemconnect. Is the "create" account part working properly and if so, is there a post detailing how to use it?
Thanks if you take the time to answer, I appreciate it!
There seems to be an issue with the new
create_claimed_accountoperation on SteemConnect. I don't know if/when they will fix that, but I will temporarily remove the possibility to use SteemConnect for creating accounts on SteemWorld.Looks like the operation is being detected correctly and even the op's data is being displayed as expected before the actual signing process, so I guess it is just a small bug on their side.
How to create an account:
The new account won't have any SP, therefore it may be required to delegate a few SP to it. Private keys are always derived from the master key and public keys are derived from the private ones. To see all keys for an account at any later point you can use the Key Generator tool on SteemWorld.
Thank you very much for taking the time to clarify that... Very helpful for sure! :)
Have a wonderful day and again you're awesome!
Sounds like a foreign language to me - but I am happy that you are doing it :) And everything you have done so far is super useful even if I have no clue how you made it 🤪
I love foreign languages, LOL!
I feel @steemchiller like Leonardo Da Vinci: he seems enlightened by a superior and vast intelligence, isn’t it?
I can’t wait to see the results.
Multi-platform service and @steemworld Open source sounds really good: to me, it could be the time to learn how to programming effectively and efficiently!
Posted using Partiko iOS
🐕 Harf harrf! 🐾
Do you understand me? 🐶
Posted using Partiko iOS
you, I do understand. Easy - either feed me, pet me, or let's go for a run!
LOL, you are an expertise dog interpreter! 👏😍
Posted using Partiko iOS
You got that right 😀👍
Posted using Partiko iOS
" ~ 0.63 times the size of the best XZ compression result."
That literally sounds like the best thing since quantum computing was introduced! WTF?
With masses of data being produced every day, such compression is going to be insanely valuable. I hope that you will be able to sell this technology to some tech giant (hopefully not google or fakebook :D) and make a billion from that.
Always happy to support SteemWorld and your work @steemchiller! This community would not be nearly as enjoyable without SteemWorld to help keep track of everything. Really looking forward to seeing what you come up with as a post viewer/editor!
=^..^=
Thank you very much for your formidable work!!
Are you half superhuman being? My brain could never creat such complex endeavor!!
Best wishes.
Hey, I just tried to use SteemWorld to claim an account. It wouldn't seem to work for some reason using Keychain on Firefox. It was just forever saying that it was writing it to the blockchain, yet nothing ever showed on the chain.
So I ended up claiming an account with Steem Connect, then trying to create the account name via SteemWorld. Had the same problem with keychain on that step.
Tried Steem Connect then, and it just said "error" every time I tried.
Eventually had to disable it too and use just SteemWorld directly.
Not sure if it was just my bad luck, but thought I'd let you know in case there's some kind of bug with how the accounts are claimed.
Both SteemConnect and KeyChain seem not to work for creating claimed accounts with the new Steem API methods that I'm using. I had hoped it will resolve itself, but that's still not the case, so I will remove the possibility to use SC and KC for account creation. Using SteemWorld directly will always work.
I come to the same conclusion as many times before in my career: Implementing external solutions will in most cases set you back and hit the quality of your own product at some point in future... Lesson learned.
btw, even though I was able to claim 2 account tickets with my valid active key. It barks at me saying I don't have authority when I try and create the accounts with my active key...And I certainly have enough credits and it is my valid active key....
Well...they could fix it eventually...but you can't really rely on others all the time. Keychain also seems to work pretty crappy with SteemPeak and a few other projects. I'm sure eventually they'll get all the bugs ironed out. Well...most of them...
I can sympathize, We all have these travails...
I have the problem with amazon MWS all the time vs. eBay and NewEggs APIs....
They don't document at all, have no centralized organized place for there published revision history and don't inform anyone in advance about up coming changes that will break your software.
I find that people designing and programming in the B.S. Agile environments like Scrum make the worst design and technical support documents, plus alot of spaghetti bloatware.
That's the major disadvantage of decentralized open source projects for the most part.
There is a lack of a uniform vision that guides the product path, and internal and external documentation is a complete afterthought. That's why it's a mess an tough to figure out most of the time. Even for a guy like me who has written case tools, compilers, worked for hundreds of companies and knows at least 100 languages and, frameworks.
Committees are great for codifying standards after a shake out.
But during design and development the old saying "to many cooks spoil the broth comes into play".
Just my sense for someone who's been in the field since 1980. Yea this old dog has some bite...
on another been working on a group app for my team....
calling it the SteemConsole here's a screen shot (pre-alpha)....about to add a second status button that opens another tab that brings up your site for the user.
LMK if you'd like to get an Alpha Copy...
Erstmal wieder herzlichen Dank für deine unermüdliche, großartige Arbeit! Hier eine kleine Ungereimtheit, die dich interessieren dürfte, zusammen mit einem Verbesserungsvorschlag:
Gestern habe ich als Stammtisch-Organisator die Autoren-Rewards für meine Einladung und für den Last Call dazu rausgesucht, um diese an die Teilnehmer zu verteilen. Da der Payout für die Einladung 8 Tage her war, also einen Tag zu lang, um in der SteemWorld in den Account Operations noch angezeigt zu werden, habe ich in steemd nachgeschaut.
Ich habe dann aber gemerkt, dass ich den Betrag von 6,036 SP doch schon zuvor aus der SteemWorld notiert hatte. Nun zeigt aber steemd stattdessen in der Auflistung 6,039 SP an, in der Block-Detailansicht aber nur den zugehörigen VESTS-Wert (sh. Hardcopy). Ich vermute, dass es an unterschiedlichen VESTS-SP-Umrechnungs-Zeitpunkten liegt, da sich der Umrechnungsfaktor ja kontinuierlich erhöht.
Nun stellt sich die Frage, welcher SP-Wert der richtige ist (auch wenn es hier keine nennenswerte Auswirkung auf die Verteilung hatte, nämlich 0,0005 SP je Teilnehmer). Ich habe den Wert der SteemWorld verwendet. Toll wäre es, wenn auch bei den Finished Posts die Payouts neben den $ auch in STEEM, SP und SBD stehen würden – und am besten auch noch die VESTS!
Vielen Dank für die Info! Ja, das hast du richtig vermutet, also die ausgezahlten 'vesting shares' ändern sich nie, aber natürlich die daraus errechnete SP je nach aktuellen Umrechnungswerten der Blockchain. Ich denke auch, dass der Wert zum Zeitpunkt der Auszahlung der wahrscheinlich 'richtigste' wäre.
Was die angezeigten Summen in den bereits abgerechneten Posts betrifft, bist du an die Grenzen des derzeit mit der Steem API Machbaren gestoßen und daher kann ich da noch nicht alle Werte anzeigen, wie ich sie gerne hätte ;)
Das ist unter vielen anderen ein Grund dafür, warum ich jetzt an einer eigenen API (SDS) arbeite. Ich habe geplant, zum Zeitpunkt der Abrechnung eines Posts die genauen Werte für diesen Post in meiner Posts-DB zu speichern. Dafür werde ich die virtuellen Operationen parsen und für jede nach 7 Tagen eintreffende 'author_reward' Operation die endgültigen Werte übernehmen.
In ein paar Wochen wird SteemWorld über die neue API laufen und die vielen Grenzen sind dann endlich aufgehoben. Auch die noch fehlenden Summen für die Allzeit-Rewards kann ich dann endlich reinbringen. Bezüglich der Account-Historie bin ich mir noch nicht sicher, ob ich das auch über meinen Server laufen lassen kann (könnte aus Speicherplatzgründen eng werden), aber es hätte seine Vorteile.
Mit der jetzigen Steem API kann ich nämlich nicht einfach die Operationen eines bestimmten Datums ermitteln sondern nur rückwärtszählend vermuten/prüfen, wo ich mich befinde, was auch der Grund für die aktuelle 7-Tage-Grenze ist. Bald wissen wir mehr :)
Danke für deine schnelle, interessante Antwort, und den 100%er!
Wenn ich das als Laie richtig sehe, müsstest du über die Blocknummern alles auf 3 sec genau ansteuern bzw. prüfen können (natürlich nur, soweit sie zur Verfügung stehen).
So kann ich die einzelnen virtuellen Operationen ermitteln (woraus sich eine Account-Historie aufbauen lässt) aber leider nicht gezielt die Account-Historie für einen Benutzer. Es würde also bedeuten, dass dein Browser alle Operationen downloaden müsste, um danach die für dich relevanten rausfiltern zu können.
Die Daten aller Steemians für eine Woche zu laden liegt im GB-Bereich und es würde die Nodes ziemlich überlasten, wenn ich das für jeden Benutzer auf SteemWorld machen würde. Der API-Aufruf
get_account_historyliefert mir gezielt für einen Benutzer die letztenXOperationen ausgehend von der gewählten Start-Operations-IDY. Die ID ist nur ein Index (der auch von Node zu Node unterschiedlich sein kann) und sie ist nicht an ein Datum geknüpft.Ich denke, es führt nichts daran vorbei, einen eigenen Index zu führen der die einzelnen Operations-IDs mit einem Zeitstempel verknüpft oder (die bessere Lösung) die gesamte Account-History per User zu speichern und mit einem Datumsfeld zu versehen (was ich wahrscheinlich tun werde).
Danke, dass du mir/uns trotz deiner knappen Zeit so ausführlich so interessante und laienverständliche Infos gibst! Mir ist da ein On-demand-Verfahren als Alternative (?, vielleicht ja völlig abwegig) in den Sinn gekommen:
Wenn ein User seine SteemWorld-Seite aufruft oder die autom. Aktualisierung ansteht, wird eine SQL-Abfrage o. ä. (ggf. mit userdefiniertem beliebigem Zeitraum, aber wohl umfangsmäßig begrenzt) direkt an der Datenquelle – also wohl bei den Nodes – ausgelöst und nur das Ergebnis zurückgegeben. Hab natürlich keine Ahnung, ob sowas möglich und/oder wirklich sinnvoll wäre.