Will Chen

This post is a brainstorm of ideas on how to focus on the core technical problems and address solving supporting technical issues as efficiently as possible. Of course this makes some assumption of what is your "core problem", but I will assume this is a "typical consumer web / mobile app". If you're a devtool company such as Docker, for example, then your core domain and supporting domains may be completely different.

• Follow the mantra "borrow, buy, or build" (I may have re-ordered it)
• Ideally, there is a well-supported open source project that solves the problem that you can use. I think examples of this are Docker, Angular, Meteor, and so on. These are all very different technologies that but they all have in common that: 1) large, active communities around them (which means you will have support when you run into issues), 2) many examples of them used significantly in production, and 3) strong core teams behind the projects (in this case, each of them have a corporation backing them)
• Secondarily, if there's a commercial product or service that addresses your need, then strongly consider using it. This is a case where you may end up using Docker the open source project first, and then realize it's very useful to use something like Docker's registry for private repos.
• Lastly, there is building your own solution. Sometimes, this means using an open source project like Docker registry but hosting it on your own physical servers or even putting it on AWS. This last one is typically about either 1) saving money (since you don't have to pay the premium pricing for a ready-to-go solution by the official provider) or 2) maximizing flexibility (e.g. add additional features, being able to switch vendors, etc.)
• Use a managed service for storing persistent data
• Use one of the big 3 cloud service providers' (AWS, Google Cloud Platform, Microsoft Azure) proprietary solutions (e.g. Aurora for AWS). You will probably pick one of the 3 providers for the majority of your cloud services anyway:
• Potentially may be the most powerful solution since AWS can optimize the hardware, networking, specifically for that DB product. The biggest downside is vendor lock-in. If AWS becomes more expensive relative to others in the future, it may be difficult to switch.
• Use a smaller cloud service focused on DBaaS such as Compose.io (although it's now part of IBM) or Heroku's PostgresSQL service.
• Databases can take quite a bit of work to setup, maintain, backup, etc. It's also the least forgiving part of your tech stack - mistakes can sink a business here. What you're really paying for, IMO, is 1) convenience - so you don't need to hire a dedicated sysadmin, 2) maintenance & backups - doing routine patches, nightly backups, essentially all the maintenance tasks that ought to do but might not know or remember to do, and 3) someone to call if something goes deeply awry (this will usually cost the most in the form of an add-on enterprise support plan).
• Use a widely-used UI framework
• Wide usage is very important because that ensures most of those nasty browser incompatibility bugs gets caught by the community and not on your product.
  
• While many people worry about having a cookie-cutter theme (especially with Bootstrap), in practice if you customized the variables, e.g. font-faces, color scheme, base font size, etc., you can still have a unique UI look for your product.
• The most interesting UI library out there is Semantic UI which uses a natural language approach and focuses on making it very easy to customize your site by swapping / combining style guides / themes.
• Use a container management service
  
• Containers have moved from the point of hype and have seen widespread production usages in many companies. Since the early days of containers, there's been many approaches to managing them, from using basic shell scripts to using traditional configuration management tools (e.g. Chef, Puppet, etc) to powerful resource management frameworks like Mesos.
• The most popular open source project seems to be Kubernetes (backed by Google which provides Google Container Engine) which was based on a decade plus of experience using containers by Google (from their legacy system Borg).
• In theory, Kubernetes can be run on any cloud service, but I think Google Container Engine will provide the easiest setup (when I tried it a while ago, it was not trivial to setup in Azure, even with all the instructions).
• Amazon has their own Elastic Container Services, which I don't know too much about, but it's a proprietary solution so there's concerns about vendor lock-in. It seems like Kubernetes has support from virtually every significant cloud player except for Amazon.
• Use a container registry service
• If you do go the container route, you will need to store all the Docker images in a Docker registry. (The size of all the images can add up quite quickly if you want to keep old versions).
• Docker and Google offer paid services.
• Use an event tracking service
  
• Services like Mixpanel are incredibly useful and provide better reporting than most homegrown solutions.
• Use a monitoring / logging service
• New Relic has a proprietary solution that seems quite advanced (focuses on monitoring availability + performance) - it's pricey but there's a free tier.
• Many companies like to use ELK (Elasticsearch, Logstash, and Kibana) for monitoring.
• Use a CI / CD service
  
• The most established services in this space are Codeship, Circle CI, and Travis CI (which has mostly focused on open source projects and recently offered private project services).
• Shippable is a relatively new comer and focuses extensively on Docker / container-related capabilities. Their pricing also seems like the cheapest at $10/container.
• Use pager duty
  
• Not sure what's the next alternative to PagerDuty, but they do a great job of creating a straightforward application that does what it sounds like.
• Use a code repository service (e.g. GitHub)
  

Other services:

• Translation management service
• Visual diff - not aware of any services / well-supported open source project for this area
• Task management (Asana, Hansoft X)
• Cross browser / device testing (Sauce Labs)

If I wanted to get something done as quickly as possible... I would use Meteor because provides the most complete solution (e.g. "full-stack" framework). It greatly simplifies tedious tasks such as setting up authentication, ensuring real-time updates synchronize properly, deploying to the cloud (Galaxy should soon make it easy even for production deployments), and building for multiple platforms such as Web, iOS, and Android.

If I wanted to make a small-scale application (e.g. internal app)... I would use Meteor because it's relatively well-used in small scale apps. If I don't expect the number of concurrent users to be that high, Meteor could likely work out of the box with relatively few performance tweaks.

If I wanted to make first-class desktop & mobile web apps... I would use React and React Native and share as much of the business logic together (perhaps using a Flux-like architecture like Redux). If performance on mobile web app was less stringent, I would consider evaluating using Meteor. Hansoft X is building a first-class desktop & mobile web app using Meteor and they used a few interesting tricks to ensure the mobile UX was optimal (read their blog post for details).

If I wanted to make a simple / relatively static website... I would avoid coding and use an out of the box solution like Wordpress + template or a full service solution like Squarespace. These tools are really easy to use and the benefit of hand-coding a static site is pretty minimal unless you're looking to create a very unique UI / UX (e.g. April Zero).

If I wanted to create a large-scale (in terms of user, complexity, team) rich web application... I'm not sure what I would use. I would first consider React + Flux + whatever on the backend (probably Node.js + Express unless there was legacy requirements). If there's real-time requirements, (e.g. seeing real-time messages from other users), I think using Meteor and investing in significant resources on scaling / load testing / etc would be the right solution. As Geoff Schmidt, co-founder of Meteor, noted, there's no easy, "off the shelf" way of building an application at "Facebook scale" without a lot of engineers working on the problem. 

Some random thoughts:

I think a year or so from now, when Angular 2 has been released and is used for production apps, it could be an interesting framework to use. For me, the benefit of a framework like Angular 2, is that it's borrowed some of the best ideas from React (e.g. unidirectional data flow) and combines it with a powerful set of supporting tools (e.g. animations, material UI components, testing tooling, etc.).

In general, I think Meteor has a lot of great benefits, many of which I've highlighted above. If I had to pinpoint, my key concerns, it would be around:

• Integrating with legacy application / services - while Meteor is composed of many components (e.g. build system, packaging, minimongo, etc.) that are in theory swappable, in theory Meteor is almost always used as a full-stack solution and there's not a lot of examples of people using parts of Meteor and integrating it with legacy systems. This isn't to say you couldn't do it, but so much of the community / documentation is focused on using the whole Meteor stack that if you run into issues swapping out the Meteor build system with gulp because the rest of your company has a shared gulpfile for example, then you may have some lonely times debugging problems.
• High scalability - If you know your application will have "Facebook scale" user requirements (which is highly unlikely, unless you're already working at Facebook / Google / tech company with huge user base, and in that case you probably have an in-house solution you can use anyway), then Meteor's lack of track record with very high scalability may be worrisome. However, to be honest, it's unlikely that your application will reach that scale. And the truth is, any system that you will create initially will need significant rewriting / finetuning to get it highly scalable.
• Testing - to be honest, I would have expected Meteor to have a core / official testing framework by now. One of the best parts about Angular was that it focused on testability from the get-go and spawned fantastic testing tooling like Protractor and Karma. However, they announced they're going to move Velocity, the official community testing framework, into core, so this will get resolved eventually.
• Hosting - I think once more information about their Galaxy service comes out, any concerns on this topic will be addressed. For now, I've read anecdotally that it's non-trivial to deploy a Meteor application, especially with scaling it out horizontally.

With the last four months of 2015, I'd like to do the following:

• Write many small applications in Go
• One big JS application
• One dev tool (hotspot analysis)
• Read more paper books
• Read the following kindle ebooks
• How to solve it
• How Linux works
• An introduction to the programming through lambda calculus

Specific example ideas:

• Simple Go web scraper
• BART real-time estimator Go App
• Relay app - measure daily activities

Tutorials:

• https://egghead.io/series/mastering-asynchronous-programming-the-end-of-the-loop

Rich Hickey gave a very interesting talk on problem solving titled "Hammock Driven Development". Rich took a much more personal (experiential) angle with this talk, so while it was sparse on academic research references it does seem to fit into at least some of what the research on brain science says. Still, while it may not wholly scientific, it was still a very thought provoking talk.

Takeaways:

• Don't code right away. Instead, start with...

1. Identify the problem you are solving. The purpose of software engineering is to solve problems, and incidentally build features to solve them. Rich advocates stating the problem out loud with your team, or even better write it down (which is pretty much applicable to every step after this as well).
2. Understand the problem fully.
1. Facts - what do you know about this problem? (e.g. what are the business requirements?)
2. Context - relevant background information. (e.g. your team uses these OS / frameworks, and there will be less friction to using something similar)
3. Constraints - implicit hard requirements (e.g. must be able to scale to 10,000 concurrent users; must be 99.99% available)
4. Are there things you know you don't know? (e.g. where is the data coming from? what happens when this API service goes down?)
5. Are there solutions to similar problems? (e.g. an open source library addresses a related use case)
3. Deeply explore potential solutions
1. What are the problems in your potential solution?
2. What are the trade-offs? (everything has a tradeoff)
3. Question marks (what do you not know yet?)
4. Flip between consuming lots of input and no input
1. Feed your Background mind: Consume lots of input (read articles, look up related problems, and critique their solutions)
2. Let your Background mind absorb: No input, just meditate, and focus on what you were feeding your mind
5. Get plenty of sleep - your brain is doing important work!
1. Get at least a night of sleep before you make an important decision

• Take the time to focus on things. You will inevitably "drop the ball" on certain things, which is OK, but tell the people important to you that you are focusing on something.
• Don't lean on the Feedback loop during development (e.g. TDD) - it's important and you will iterate it, but rely on it instead of focusing on the previous steps.
• Don't be afraid to make mistakes, requirements will change, you will come up with a better idea... it's OK

https://www.youtube.com/watch?v=f84n5oFoZBc

• Read a book for at least 30 minutes - (better if it's dry)
• Meditate for 10 minutes
• Write down any thoughts to the iPad for 15 minutes