Aether News & Updates - July / August 2018

Hey folks - this is the Aether monthly. This one is pretty exciting with screenshots, and it is probably going to be the last pre-launch update, because it’s about around one week to some semblance of code complete. The app is running fine on my machine now, with all features built for the backend and frontend, and most for the client (user interface) side. There’s a few screenshots of the current state below. None of those are design mocks, it’s the real thing, pulling data from real other nodes. (The data within them is auto-generated for load testing, so there are no users on the system right now.)

Recap: I’m new. What’s this thing?

Hey there - welcome in! There was a write-up on that last month, do check it out here.

July updates

Javascript

July, it was mostly about going deep into the Javascript rabbit hole to build the client user interface. (Quick recap: Aether has three parts, client, frontend and backend. Client is UI, frontend is the graph compiler, and the backend is responsible for distribution of underlying entities and communicating with other nodes.)

The tl;dr is that designing and building user interfaces is fun —to me, at least— but doing it in a world where everybody wants you to use Webpack in their own way with slightly different and subtly incompatible configuration is not.

Packing, unpacking and compilers

There is one general rule that is useful in building most things: to do the simplest, dumbest thing that works. I heard it called the Linus’ rule, though it is not entirely clear if Torvalds has ever said it. (Nevertheless, the Linux kernel and its success could be considered a manifestation of this idea.)

Go works well for this mindset, and that’s why the frontend compiler and the backend is written in it. It’s a place where you can actually control the layout of your structs bit by bit in the memory, and it does a great job of getting out of your way, in that the abstraction of Go is fairly thin: you are working with a CPU and a memory that you can put bits into, and there is a small, helpful, and fast garbage collector provided if you want to use it. That means, in Go, you can have as few third party dependencies as possible, and for the things that do not exist in the standard library, you can efficiently build what you want. It also helps that the core language is very small, and there is usually one way to do things. This is very useful when you actually want to read code on Github on how libraries implement a specific functionality. Everybody has to write it mostly the same, so library authors aren’t writing what is effectively a completely different language. coughC++cough

With this in mind, I opted for Vue as the base for building the client. This was largely motivated by my desire to avoid Webpack, and this whole seemingly-always-half-broken part of Javascript ecosystem that frequently picks a favourite son every 6 months and kills it with fire shortly afterwards. Considering past experience of projects that had to be maintained that were still on Grunt (remember that?), others still on Gulp, going into this again with Webpack did not sound enticing to the least bit, especially considering that the main selling point of it (single Javascript file) is not very useful in a local Electron environment. To that end, Vue offers an ability to use it through the old-school way, which is to include it as a `````` file. Very simple! Since my use case did not need Webpack, this made Vue a first and only choice. Much later on, and long beyond the point of no return, however, it became obvious that this was a siren song.

The issue was this — Vue considers the ability to split your code into multiple Javascript files (single-page components) an advanced feature. And the moment you hit this, which is early enough that your project is not yet functional, but late enough that you can’t really back out, Vue tells you that you have to use Webpack.

This was a huge disappointment, since the majority of the reason for using Vue was to avoid Webpack. It’s not that one cannot get it to work, it’s just that it does not make sense to integrate with yet another build system, especially in this specific case, one that offers exactly zero benefits. Ostensibly it makes things easier for library developers (since they can offload a decent chunk of ‘getting things to work’ part of library-building to Webpack, which, promptly, unloads it onto you), but it would have been very nice if Vue had provided a way to use single-file components in a way that does not require a build step, especially considering even Angular-1 in 2014 had a way to do so, in the form of ng-include directive.

(I know that it is not the fastest, since fragments have to be all loaded before the page needs to render, but in this case, the files are already available on the local filesystem, thus it does not matter.)

Nevertheless, it was a good few days spent on getting Vue, Webpack and Typescript working. To be fair, Vue is awesome, Typescript genuinely fixes legitimate problems with Javascript, and Webpack can help under right conditions. It is a good thing that all three exists. It’s just that it is a little sad that the Javascript environment is such a high-churn, high-thrash place, and coming from 4-5 months of writing Go code where the language is sound, the libraries, the very few you need, are of excellent quality, and there is only one specification of the language, one gets to realise why the concept of Javascript Fatigue exists.

I do not want to belabour the point. The one last thing that is quite interesting is that the main three Javascript libraries that is used (Vue, Typescript and Webpack) don’t even agree on what is syntactically valid Javascript. There are multiple specs of Javascript floating around, and there is Babel, which is a library that compiles new, not-yet-ratified (often, not-yet-finished) Javascript language changes into old, mostly-valid Javascript that you can run. It is a fun tool to test new syntaxes. Unfortunately, it appears that everyone is so sick of old, standard Javascript that it is a dependency on almost anything, so that library authors can write new, shiny Javascript.

Considering the fact that it implements changes that are drafts, and drafts sometimes do change, the tendency to use the newest, cutting edge, unstable Javascript features ends up in a place where the documentation on the Internet is written in Javascript that you don’t understand (because it was created literally a week ago), that does not work, that does not compile (because the spec changed two days ago), and most importantly for most intents and purposes, of negative actual value.

If you are a library author, or if you are writing documentation of any form, please consider writing your documentation in good, old, standard Javascript that actually executes without requiring a compiler set up exactly the same way you have on your local machine.

Regardless, it should be obvious where this is going at this point. Fast forward a month of writing Javascript, and there is more than one million lines of Javascript in the node_modules directory, totalling 2+gb of code that there is no actual dependency on, but is there nevertheless due to how node ecosystem and Webpack in general works. Thankfully, the final end result is only a few megabytes of front-end code, since that is the actual code that is needed. The rest is code that Webpack refuses to compile without, but far as one can see, none of it does go into the final end result.

The fundamental goal was to achieve was a client that was simple to write, simple to understand, and simple to maintain. Unfortunately, it appears that this is close to, in the Javascript world, a yet-unsolved problem. There will be some maintenance to do when Webpack goes the way of Grunt and Gulp. Regardless, I’m happy with the client code given the constraints, and it is as simple and maintainable as technically possible. The hope is that it will only improve.

WebAssembly is something to watch, because it does allow one to completely replace Javascript and its dependencies with another language of your choice (which you can do right now), and have it be able to interact with the DOM (which you cannot, and won’t be able to for the next few years). Since what matters is the DOM access, whenever that comes, it will be some welcome competition to Javascript and its environment.

August updates

Current status

As of today, the app is about a week to code complete, there are a few pages that need to be designed and implemented (notifications, status, home/popular views), and a few components that need to be finished (post and thread ranking logic based on signals). I’m also getting an Apple Developer Key, so that in OS X, the app will open without scary warnings, and I will be able to push auto-updates in a way similar to Firefox does. (Download update in the background, install on next start, with an ability to disable it).

I will also be putting out a Discourse forum for user support coincidentally, so that there will be community support available.

Integration testing

The expected code complete is the end of this week, but given the prima facie impossibility of giving accurate estimates in building anything of nontrivial complexity (estimating the task takes full knowledge of the task, and in building tech products, the task itself is 90% getting the full knowledge of the task at hand, and only 10% building it), some skepticism is probably warranted.

Regardless, the goal is to have a pre-release executable for OS X (only because I develop in OS X. The goal is to provide Windows, OS X and Linux versions in general availability, same as Aether 1) as a test to see network function and find bottlenecks, if any. It will be followed by a beta release that is generally available, and from that point on, the auto-update system will keep your app up to date.

In short, by the end of August, the network should be transmitting data in some form.

Feature freeze

A feature freeze is a set date where a product stops adding new features before a launch date. This is an useful cutoff, because trying to cram as many features as possible into a launch can mean biting more than you can chew. There are some features that did not make it to the feature freeze deadline of August 1: the feature set that allows for ongoing moderator ballots in communities. These features are code complete for backend and frontend, but they will ship disabled in the first version until the client (UI and product integration) is complete. After the product stabilises and the network is operating correctly under real load, this is what comes next. The expectation is that they will make it into the first non-beta release.

Code freeze

A code freeze is when one stops writing any code and focus on bug fixes, finishing existing features, stability, and QA. The pre-release version will be the public QA test, and if everything goes right, the external QA test will replicate the results of the internal one that I’ve been running on my own local network. The rough code freeze date is August 15.

Pre-release

After the code freeze, the tasks that remain are those of packaging the app into an executable, updating the website, and setting up a bootstrap node so that newly downloaded nodes will have a point to bootstrap from.

Beta launch

This will happen when the pre-release gives enough confidence that the app is stable, and can handle nontrivial amount of users. Here, the major thing to keep an eye on is not that the app would not work (it does now, and it likely will continue to do so), but it is that the network is slow to deliver every post into every participating node. Since this requires fiddling with network variables (such as how often a node checks for updates in a minute) and is untestable in a synthetic environment, the only real way to know this is to actually have real people using it, and getting the real-world data on dispersion speeds.

From this point on, it gets a little hard to say, because it depends on the reception of the app, and the real world usage. If this app becomes a place that we all can enjoy spending our evenings in, I personally would consider it as success, even if it does not succeed in financially supporting me working full time on it. That said, the ideal path that I have in mind is that this would support me and a small core team (which does not yet exist, it’s just me, for now) financially enough that we can devote our full focus to offer a mass communication platform that is free of the catches of what’s available today. After all, Aether offers what they won’t sell you at any price: your privacy.

Screenshots

Not mocks, actual app! 🙂 (It goes without saying that these are all works-in-progress.)

Addendum: Who’s funding you / this work?

I’m currently working on this project full time, and I have been doing it for the past six months. I’m funding it through my own savings. My initial plan is that I want to release the full app, and then consider getting a job, because, well, my savings aren’t infinite. That lines up nicely with the end of the majority of the development effort for the release, so for the initial release, I don’t foresee any problems. After that, it all depends on how many people actually end up using it. If a lot of you guys do, then we’ll find a way to make it work.

My motivation is not that there’s money to be made out of this. If you’re interested in that, ICOs might be a better bet, but I’d still recommend index funds over anything crypto-related, Vanguard is a good provider for that. I find most ICOs to be fairly distasteful (with some very rare exceptions), because I’d rather have my work speak than empty promises, as opposed to what ICOs typically do, hence my starting to speak about my work just now, when 80%+ of the total work is already done and implemented in cold, hard code.

If you are happy with any of the existing solutions solving similar problems out there, you should consider donating to them. There are a lot of open source creations for different kind of needs. They do often suffer from poor user experience and poor usability though, most of the time, which unfortunately renders them unable to gain adoption. If your project requires the user to compile it before running it, it’s fair to say that your audience is not the general public. That is in itself not a bad thing, and if any of those projects actually satisfy your needs, consider donating to them. I’m focusing on better experience for the mass public, to provide all of us with more accountability, more transparency, more privacy, and hopefully some real discussion that we all sorely need — as a human society, we’ve gotten far too good at stifling each others’ voices.

The reason I think I can make a difference is that I’m an ex-Google, ex-Facebok product designer with a lot of experience in the field, and while I have some ideas on what can be improved, I’ve learned through the long and hard way that the best ideas come from people who use the actual thing. I’ve also learned a second, almost as crucial other thing: people doing something that they genuinely, honestly care about tend to achieve much better results than a team of highly experienced specialists that are just meh about it, even if the first team is relatively inexperienced. I do have experience, and I do feel this problem every day, to the point that I was willing to quit my job and start working full time on it using my life savings. I’m lucky to be able to do that, but more than luck, I also care enough about this that I wanted to do that.

In summary, I think there’s a need for it, and it should exist, in some form of another. Nobody else seems to be doing it, so I’m doing it. This won’t make you rich, this definitely won’t make me rich, but this has, if done right, has the chance to create an open, simple, free, decentralised, and private way of mass-communicating for the next half century — a public online gathering place that also happens to belong to the public, with no catches, and no kings.

If you think this is worth supporting, you can fund me via Bitcoin at: 1K12GwzAtPWa6W864fkm48pc4VNqzHXG4H, and via Ethereum at: 0x3A4eC55535B41A808080680fa68640122c689B45. Your funding will extend the amount of time I can stay working full time on it. I don’t need the funding to finish it (I made sure that I can actually finish it with my own savings), and if you are a student, new graduate, or on a budget, you should not — keep your money, use it for yourself and use your money for good whenever you can. But if you consider yourself relatively well-off and thinking about how your money can do the most good in the world, and you’re interested enough that you’ve read this far, I’d offer this as a pretty good bet.

I’ll open a Patreon page, eventually, but I’m focusing on actually writing the code and releasing the app right now. The Patreon benefits I have in mind are mostly cosmetic, such as getting a ‘benefactor’ username class with some visual distinction, and priority in picking from available unique usernames. If you fund me through Bitcoin or Ethereum, you’ll get the same benefits as Patreon supporters.

Lastly, if you’re in San Francisco, and you’re excited about this, have questions, hit me up, always happy to go for a coffee. Might take a while to schedule considering that I work 14-16 hour days, but we’ll probably find a time.

As per usual, feel free to reach out to me with any questions at burak@getaether.net.