SAFE Network Dev Update - May 16, 2019
Summary
Here are some of the main things to highlight this week:
- We made various changes to the safenetwork.tech website and added a new Roadmap page.
- RFCs for new network data types have been published today: RFC 54 and RFC 55. See this forum topic for an overview of what is it about these new data types that makes them different and how they may impact current SAFE APIs and applications.
- The Secure Message Relay RFC has been published: RFC 56.
- @ fergish released a new podcast with Spandan about why we’ve made the move to incorporate QUIC-P2P.
- We put out a couple of Medium posts on the Fundamentals - you’ll find Part 1 here and Part 2 here.
- The next edition of the SAFE Network: Chicago meetup will be on Thursday, May 30th.
- We published a patch release of the SAFE Authenticator mobile application.
- @ Viv, our current CTO, has decided to step down from his role at MaidSafe.
UX and Website
Those of you who were quick off the mark will have noticed that we’ve just rolled out an update to the safenetwork.tech website.
It’s been bubbling away alongside other work for some months now, so it’s great to have it out in the wild. Think of it as an evolution of the site and brand, as well as bringing out some of the original intent of the site with a bit more focus.
We’ve sharpened up the typography, refined the palette, simplified the grid, and added a little more of a flourish to the homepage.
While much of the content remains the same, we’ve taken the time to overhaul the How It Works page, improved the on-boarding process when folks are ready to Get Involved, and of course can introduce the anticipated new Roadmap page.
The Roadmap page itself reflects much of what was revealed in last week's update, but we hope it lays things out in a more digestible, accessible way. It’s a living document that will be updated continually as the project ramps up towards beta.
Hopefully, it will be helpful for you all to have a one-page overview of all the moving parts as they start falling into place; easy access to the relevant RFCs, discussions, and designs; as well as having the pleasure of seeing the green ‘completed’ ticks appearing!
These site launches are a big undertaking, with lots of work from across multiple teams; so thank you all who contributed, in particular @ shankar, @ dugcampbell, @ JimCollinson, @ Karamu, @ SarahPentland, and @ nadia. That Oxford Comma was just for you Nadia.
As always we appreciate your bug and typo reports, drop them in as a GitHub Issue if you sniff one out. If you don't use GitHub then feel free to add a comment to this post and we'll pick it up.
Marketing
Required listening for all this week as @ ustulation was invited back onto @ fergish’s excellent SAFE Crossroads podcast, this time for a conversation about why we’ve made the move to incorporate QUIC-P2P within the Network. We've also been focusing heavily on the SAFE Network Fundamentals this week on Twitter - just search for #SAFEFundamentals. We also put out a couple of Medium posts on the Fundamentals - you’ll find Part 1 here and Part 2 here. Just to recap: we're republishing (under the medium-repost tag) so that those who don't like Medium don't need to visit it. However, for everyone else - please do continue to click through and support with claps on Medium as this helps the distribution of our messages!
Quick comment on exchanges: Bitker is currently waiving fees on MAID trading so worth a look. And a final reminder about the upcoming meetups: @ ustulation and @ nbaksalyar speaking at SAFE Network: Brighton on Thurs 23rd May, @ nbaksalyar sharing the stage with Ruben Verborgh of Solid fame at the first SOLID London meetup on Wed 22nd May. And last but by no means least, @ sotros25 is doing yet more sterling work, running the next edition of the SAFE Network: Chicago meetup with the Chicago Blockchain Project on Thursday, May 30th. Thanks in advance for all the work throughout the community helping to publicise each of these events!
SAFE API & Apps
Several of us were focusing on reviewing several aspects of the upcoming new data types, including the AppendOnlyData data type needed to have a perpetual web! This would have an impact on some of our implementations on the front-end side, especially around the new APIs required to support it, and how some of our applications and data representation would need to be adapted to this new data type and the whole concept of perpetuity of data.
We therefore started a new forum topic where we highlight, as an overview, what is it about these new data types that makes them different and how they may impact current SAFE APIs and applications. Please join us in this discussion to clarify doubts and/or expand on any of the aspects touched upon.
This type of research tasks will for sure continue in the next few days and weeks, along with some effort to both update some of the already published RFCs, or to create new ones like the one we are working on for a SAFE data CLI as briefly mentioned before in these dev updates.
The browser has finally gotten some cleaner/more flexible tab and window data structures. Which have solved a raft of wee browser niggles as well as opening the door for some more standard browser features later (e.g. tab reorganisation/dragging). We've also gotten one major step closer to a clean and shiny codebase, with all browser code linted and CI tests for the same. Otherwise we've been investigating if automated website tests might be possible with the SAFE Browser and TestCafe (which, while looking much more positive, is still proving to be an elusive dream). Auto-updating browsers are on the way too! Initial drafts are working well on macOS, but need some further digging for Windows and Linux.
We published a patch release of the SAFE Authenticator mobile application earlier this week. It includes a fix to prevent the Authenticator app from crashing when sending the response back to the app and the app url is not found. Also, now the authenticator identifies an invalid container name in authentication and container requests. And it provides information whenever an app requests access to another app's container. For more information have a look at the changelog.
SAFE Client Libs
Over the past week, we have been focussing on the RFCs for a major change in the network data types. The RFCs have been published today: RFC 54 and RFC 55. With that, we'll finally adhere to the network fundamental of data perpetuity. With all of the community feedback taken into account, we decided to address it without breaking the compatibility with our existing APIs, but by extending them instead.
There are several major changes: first, we are looking to introduce a notion of unpublished and published data. Unpublished data is any kind of data that can be read only by the owner (or multiple owners) or by a private group which has read permissions. Published data is stored in perpetuity and it can never be deleted or changed--even by the owners. Thus, with this approach, the well-familiar MutableData will be available only for private use, as an unpublished data type. Second, we're looking to introduce a new data type, AppendOnlyData. While it's also a key-value store, it will have different semantics from MutableData: instead of changing the existing keys, new entries need to be inserted to change the data. But even with these changes, all past entries will be always available.
This planned change is big, but it opens up a lot of possibilities in the network. For example, you can browse all past versions of a website--just like on the Internet Archive, but for the entire Web. It also addresses the problem of link rot, which is also very important for Linked Data and the true Semantic Web.
The RFCs are not final and we'll be looking for your comments and opinions. We also plan to start implementing some proof-of-concepts in parallel, so these are exciting times for the Client Libs and the SAFE Network in general!
Work on the PoC for RDF support in SAFE Client Libs also continues in parallel. We have implemented Rust bindings for the Redland API to parse RDF documents into RDF graph objects and vice versa. Initial implementations for SPARQL querying on RDF graphs are showing promising results. We are currently implementing a proper approach for graceful error-handling. We have also integrated some RDF functionality into SAFE Client Libs that enables storing, retrieving and updating RDF graphs using Mutable Data. We will soon plan integrations with the upcoming data types too!
We'd like to sign off by welcoming @ marcin who's now back in action after a well-deserved vacation!
Routing
Implementation of initial aspects of Fleming and work on design continues.
In Routing, we created a few PRs for some code cleanup and further preparations for the implementation of Node Ageing. These improve our tests and our handling of candidates attempting to pass the resource proof - as the candidate's state should be shared between the members of the section, changes to it should be a result of PARSEC consensus, and this is what PRs 1628 and 1629 aim to achieve.
There is also ongoing work on a bug in PARSEC uncovered by Routing tests some time ago. This bug proved to be very elusive and difficult to identify precisely, but we believe we have finally found the cause and are working on fixing it.
On the design front, we had some internal discussions on RFCs about Safecoin and Message Relay that are in the making. Some issues have been raised and we are fixing them. The Secure Message Relay RFC has been published today, and the Safecoin RFC will be following soon.
We also continue our efforts on designing Node Ageing flows in routing_model. There are still some missing pieces in the flows that we are continuously working to complete - a list of these can be found in Issue #7.
QUIC P2P
There are more things to work on in quic-p2p, but now that we have released the first public crate version, we'll be slowing down the development for the time being, putting our resources to more critical things.
Viv
Next, we have a piece of sad news that we'd like to bring to your attention. @ Viv, our current CTO, has decided to step down from his role at MaidSafe. As everyone who's followed the project knows, Viv’s contribution to the company over the past 8 years has been exceptional, having worked his way up from a junior development position to lead the development teams. He has been an invaluable member of the team and he will be missed.
There has been internal discussion around this topic for a while now and given his commitment and belief in the project, Viv has not taken the decision to leave lightly. However, after lengthy conversation, he's of the opinion that stepping down will enable the executive team to become more streamlined in design and project decisions moving forwards.
Viv will continue to work for MaidSafe until the end of July to ensure a smooth transition to @ dirvine who will initially assume the role as acting CTO. We don’t plan on hiring a replacement straight away and will take some time to consider the next steps, which could include promoting from within the company.
It’s been a lengthy journey that we’ve been on and it's always sad when someone leaves the team to return to the community. But this project continues to be built on the hard collective efforts of many and we're always so grateful for everyone that has put their shoulder to the wheel and put in a hard shift, particularly when it comes over the course of many years - and that very much applies to @ Viv.
Project Management
Finally, with @ dirvine taking over as acting CTO, we’ve reorganised slightly in order to facilitate project management taking a much more active role in delivery.
We haven’t hired new people, and we aren’t taking dev time away, but we have worked on clarifying some project-related responsibilities for some staff to help us ensure we deliver our goal. The Project Manager has a support team of two (ensuring front and backend have dedicated focus), three staff have project coordinator roles to ensure tasks in the plan are correct, have reasonable estimates and the priorities are correct.
We also have a ‘strikeforce’ team of four (two for frontend and two for backend) who will use their knowledge and experience to work with the teams to continue to improve the delivery quality in addition to providing support to the teams when tasks are hitting issues or looking to overrun.
The RFC process will come under far greater focus as part of the design process. For transparency, we now have a private Pre-RFC repo where these discussions take place internally before we make them public. This ensures that the team can be extremely focused on ensuring that the topic has been interrogated deeply before the draft RFC is made public - at which stage we all, as a community, can debate things thoroughly so that we collectively end up with the most comprehensive solutions.