Link Post – Scraping Kindle Highlights –

Scraping Kindle Highlights –

Using a combination of Python and the Python libraries Requests and BeautifulSoup, it’s entirely possible to write a Python script that will log into Amazon, get a list of all of the books on your account, and download the highlights for each

This is exactly the kind of thing I’ve been looking for ever since Shaen Blanc and Ryan Holiday wrote about a commonplace book. I will see if I can pipe this straight to DayOne.

Safari Suddenly Updating Won’t Change Anything for Web Developers

There was some recent buzz recently in the web community about a post from Nolan Lawson entitled “Safari is the new IE6”. Then there was a counter argument, that the problem is a lack of browser competition on iOS. The reality is more nuanced, and the solution is “That’s just how web development is, use a polyfill and move on.”

Lawson mentions, a resource that web developers (such as myself and Nolan) frequently reference to see how supported the latest browser features are.

My interpretation of the Can I Use data is a bit different from Mr. Lawson’s.

I m going to look at the technologies mentioned specifically in the post:

  • IndexDB
  • Shadow DOM
  • Web Components
  • Service Worker
  • Web Manifests (Offline Web Apps)

First, there was a lot of talk around IndexDB and the broken implementation in Safari 8.3.


And while it would be nice for Safari to support the feature, even if Safari suddenly supported the spec fully tomorrow, according to this graph, you have a larger number of Android Users (Android Browser 4.3 and below and UC Browser for Android) that don’t support the feature at all.

Looking at the Shadow DOM:


Its supported by Chrome, Android and Desktop Opera (Which is using Chrome’s Blink rendering engine, so it gets it for free). Again, if Safari implemented this, you would still be missing IE, Firefox, and UC Browser or Android, and Android Browsers below 4.4 (This is sort of a pattern from here on out, so I will only just put the screenshots).



“Web Manifests” are a special case in that Can I Use doesn’t actually list “Web Manifests” or “Manifests”, but when I search for “Manifest” I get a single result, “Offline Web Apps” which is actually mentioned by Lawson. What does this browser support look like?


Wait…so Safari actually supports this technology? but not “web manifests”? So we Google “web manifests” and the w3c has a Working Draft. It has a big red warning on the page.


So, there is already a way to do this in Safari, but Chrome does it differently, and the spec is still unstable. As a matter of fact, Service Worker, Web Components, and Shadow DOM are also in a state of W3C Working Draft.

As a matter of fact, Custom Elements (another way to say “Web Components” and Shadow DOM both have this warning across the bottom of their working drafts:



The only item on our list that is in the state of W3C Recommendation is IndexDB, which hit that state in January of this year.

I have been a web developer for 15 years, and polyfills, shims, and abstraction layers are par for the course, especially when dealing with features at the cutting edge of the web. Hell, I still have to support IE8 in almost all of my projects! Scroll back through those graphs, and you will see IE8 and 9 are red in all of them.

It honestly feels more like Lawson just wishes Safari was Chrome. This isn’t an attack on Lawson. He actually wrote a follow up post in which he says

I tend to write about weird esoteric stuff like IndexedDB and WebSQL, maybe throwing the normals a bone with something about CSS animations.

Its important to note that when he writes about “esoteric stuff” and “normals” he means esoteric and normal among web developers. Layman “normals” are not reading about CSS animations. He is an admittedly bleeding edge developer lamenting that he doesn’t have the latest toys in Safari like he does in Chrome, and it was all taken out of context. He writes Android and web apps and probably writes his apps in Chrome first (So do I, and Chrome isn’t even my default browser) and is probably annoyed when things don’t work properly in the other popular mobile browser. (Although it probably doesn’t work on a large percentage of Android phones either)

Apple has never been a company to implement a cutting feature unless it will massively improve the user experience (like killing flash and pushing web apps for iOS). Remember that they took the wait-and-see approach for 3g, LTE, NFC, Large Screens, the list could go on and on. But just like hardware, Apple isn’t going to implement a new browser feature immediately because it’s new. It will generally wait until the tech is mature, it can manage battery life, know that the spec is stable, and browser speed and then implement.

The one place where I do think Apple can do better is in sending representatives to the W3C to influence the specs. When you look at the Specs and Working drafts mentioned above, you will notice that Google has people on every single spec (sometimes they are the only author), so of course Google is implementing the working draft spec, they wrote it! But that’s not Google’s fault. Apple should be in there as well, writing the spec they want to see with them.

**As a side additional note, the UC Browser on Android proves that opening up the iOS browser eco system is not a solution, you could just as easily wind up fragmenting the iOS rendering engine even more winding up with several users on iOS that don’t support features that are implemented in Safari.

Building a Posterous Clone with Node.js

This is the first article in a series where we will develop a Node Js project from beginning to end.


Why Node?

Node has the primary advantage of being fast. I mean REALLY fast. This is because it is asynchronous by default. Programming asynchronously is also what will make Node challenging. You need to start thinking about whether or not you need one function to complete before calling the next one, because this will dictate how you structure your code. We will cover this more in-depth in another article in this series.

What Else?

I have decided to go with the MEEN stack: MongoDB, Express, EmberJs and NodeJs. This has a few interesting advantages:

  • This stack is JavaScript top to bottom from the database, to the backend framework, and your front-end framework, all runs in JavaScript
  • This stack is fast, MongoDB is an incredibly fast data store, Node is a very fast backend, and as a web framework, Express is very minimal and fast, and Ember, once loaded is an incredibly fast front-end.
  • Because everything is JavaScript based, everything deals with JSON as completely native (even your database)
  • As a datastore, MongoDB is very flexible. You do not need to create collections to store data, just store data to it, and MongoDb will create it on the fly. Need a new field? just add it to the next thing you save.

We will also be looking at using a job queue server (most likely Gearman) to manage backend processes, and learn why this will make our application even more scalable.

That Sounds Cool! So What Are We Going to Build?

We will be recreating the now defunct Tumblr clone Posterous, that was bought and subsequently shuttered by Twitter.

Like Tumblr, Posterous was a mini-blogging platform with an emphasis on Post-types (video, audio, link, quote, text, etc.) and a community. It had two interesting features that set it apart:

  1. It was based around posting by email (originally)
  2. It aggregated full content to just about EVERY service.

That second item in particular is what will make this project very exciting. We will upload photos to our new app and then propagate them out to Facebook, Picasa, Twitter, Flickr, etc. Videos will go to Facebook, YouTube, Vimeo. We will repost everything to WordPress Blogs, Tumblr, etc. We will be using Node to connect to all of these different APIs and learning how to work with them in Node.

Ok, But the Big Question Is….WHY Are We Doing This?

In my work, I help startups develop product concepts from beginning to end, I architect solutions, I make sure these things can scale up to handle millions of users, and of course, I build these products for those startups as well. Its an involved process, that I love going through, and I want to be able to share that process with everyone else.

Up Next…

In our next article, I will walk you through setting up our server, and getting a simple web server running with Node.Js and Express.