Eamonn O'Brien-Strain

Mastodon – Bluesky – Threads

One of the purposes of this blog is to experiment with simple ways put up a blog on the web without depending on any blogging platform. For this I wanted a free way to deploy static web pages on my own domain name. For this I chose Firebase Hosting, partly because I am very familiar with it, having worked on the Firebase team and personally knowing many of the Hosting development team, but also because it is actually the best solution I know of for what I want, and it is free for sites that have the traffic of typical personal sites.

People who are comfortable using the command line might be interested in following along the steps I took.

  1. Go to http://firebase.com, login, and create a project.

  2. Install Node. If you don’t already have it installed already, I suggest first installing nvm then doing:

    nvm install stable
nvm use stable</pre>

  3. Follow the instructions in the Hosting quickstart.

  4. Replace index.html with the HTML you want as your home page.

  5. Do firebase deploy

  6. I’d recommend storing everything in Git, and you might as well make it a public repo, as the web site is public anyway. (Mine is on GitHub)

  7. You also might want to register a domain name, and use Firebase Hosting to use that domain name (complete with a free SSL certificate so you the https: security for free.). I did this for eamonn.org

On Twitter yesterday I made a big announcement that I’m moving to a new team within Google.

For most of my five years at the company I have been working on products for external developers, first for App Engine where I lead the memcache team, then for Firebase and Cloud where I worked on privacy, security rules, and serverless events.

But now I am about to move to the team responsible for the main Google Search user interface. This is a huge change. I’m moving from infrastructure engineering to front-end engineering. And instead of building products for millions of developers, I’m building products for billions of end-users.

And this seems like a good time to make yet another attempt to start writing a regular blog. I’m going to start by eschewing the complexities of modern web development, using simple, plain handwritten HTML files. Now that I’m a front-end developer again, I can practice my skills here.

“History does not repeat itself, but it rhymes” Around the turn of the 19th Century: the French Revolution and the American Revolution, together with the Industrial Revolutions killed off the ancien régimes with their aristocrats and divine right of kings.

The land-owning aristocrats lost their power and sometimes their heads.

Just after the turn of the 20th Century: World War I, together with the Bolshevik Revolution and the Scientific Revolution killed off a century of stable interest rates that had allowed the wealthy to live a life of leisure.

The capital-owning wealthy had most of their wealth wiped out.

Just after the turn of the 21st Century: resurgent religious fundamentalism and right-wing nationalism, together with the Internet revolution may be about to kill off a world in which multinational corporations and a technocratic elite thrived among increasing inequality.

What will happen to the technocratic elite?

fissioning HP logo

What is wrong with HP that would spur its announced fission into two daughter companies? My view from the technology trenches is different from what I am reading in the business press.

HP is not really a technology company any more. It is a logistics and marketing company. Much of its engineering is outsourced. I am not surprised that it is splitting itself according to its go-to-market and channel criteria, not technology criteria.

HP does not know how to innovate. Instead of letting a thousand flowers bloom and leaving itself open to positive black swans, it doubles down on big bets. When I was at HP Labs, we were encouraged to only propose projects that promised a billion dollars or more in revenue. There was no room for the small speculative project.

Software is eating the world, but HP cannot write software. It is even confused at the word “software”, using it as the name of a division that is just 10% of the company, even though software is actually crucial in every one of is divisions. HP’s software development is incredibly fragmented, with no returns to scale. Each of its many software development groups scattered throughout the company have their own idiosyncratic software engineering practices, some good some bad. The code of each development group is walled off from other groups, so code sharing across the company is rare. Managers regard access to their organization’s code as a weapon to use in internal battles and negotiations.

HP cannot attract and keep good software engineers. For example, I personally know 24 other engineers who, like me, have made the migration from HP Labs to Google over the years, and there are probably many more. In contrast I don’t know a single person who has moved in the reverse direction from Google to HP. Some of this is no doubt driven by superior remuneration and perks, but I think what really clinches the deal is that in a real technology company what the engineers do really matters. For that they are willing to tolerate the packed elbow-to-elbow work areas and the straitjacket of a company-wide unified software development process.

Maybe HP does need to split, so that at least one part of it has some chance of turning itself back into a real tech company. I wish my friends still there all the best as they navigate the transition.

Map of the world show areas of with more than 5 people/km²

The political map of the world is a familiar image, and we have become used to the shapes and relative sizes of the countries. However the map gives undue weight to some countries with large uninhabited areas, even when you choose a map projection that does not distort areas too much.

So a few weeks ago I tried playing with a world map to give a clearer idea of where people actually live. I chopped out the parts of the countries that have very low population density, less than a threshold of about 5 people per square km (13 per square mile), To an extent this gives makes countries of equal population have more similar areas than a standard map, while stopping countries being distorted and sliding about as they do in a cartogram.

And looking at the countries through this lens is an interesting exercise. Let’s take a tour …

Australia

trimmed Australia Australia is one of the most noticeably diminished countries. You can see that it has a relatively small population for its size and most of those people live along the East and and South-East. There is a little outpost of Perth far to the West, and if you look carefully you will see the tiny dot of Darwin in the North.

India

trimmed India India has more than 50 times the population of Australia in less than half the area. Not surprisingly it is packed. It seems nowhere is uninhabited except maybe a little patch in the North in India-controlled Kashmir along the Himalayas.

Kazakhstan

trimmed Kazakhstan In this view, Kazakhstan is also much diminished and looks like Australia, with large uninhabited areas in the center and with most people living around the boundaries of the country, either in the Steppes along the Northern border with Russia or along the South-Eastern border with the other ‘stans and China.

Ireland

trimmed Ireland The Republic of Ireland is almost all above the density threshold with just a little erosion along the West coast in Kerry, Galway, and Donegal.

United Kingdom

trimmed UK England and Wales seem mostly intact, but Scotland has lost much of its Highlands.

Canada

trimmed Canada Canada is another country that is greatly diminished in this view. Despite its vast Northern expanses almost all its populations are huddled in the South along the US border, a lot of them down near Toronto and Montreal.

United States

trimmed US The USA has a very noticeable vertical line down the center of the Great Plains splitting drier (emptier) areas like the Dakotas and West Texas from wetter (fuller) areas like Iowa and Eastern Texas. Large parts of the interior West are blank. Alaska has almost disappeared, the largest remaining area being a small blob around Anchorage where most Alaskans live. Other than Maine, the Eastern states are largely intact except for some patches in the Appalachians and some large parks.

Mexico

trimmed Mexico Mexico is pretty sparsely populated towards the North along the US border.

South Africa

trimmed South Africa South Africa has large sparsely inhabited areas, especially in the West, so that Cape Town in the South West is quite isolated.

Russia

trimmed Russia Russia is spread out horizontally along the Steppes getting less and less populated the farther you go West. There are just a few tiny outposts along the Pacific. There is also a southward extension into the Caucasus region. Like Canada, its vast Northern expanses are largely unpopulated,

Egypt

trimmed Egypt Egypt, like many other North African countries has much of its population concentrated along the Mediterranean coastal plain. However Egypt also has a dense population concentrated along the Nile river, and also some people living along the Red Sea.

China

trimmed China Although China has a huge population, you can see from this map that almost everyone lives in the Eastern half. The Tibetan areas are very lightly populated, and there is a Western outpost in the Turkic areas bordering the ‘stans.

Seamus Heaney Obituary in the New York Times

Today the New York Times marked the death of Seamus Heaney with an above-the-fold front page photo, and a long article illustrated with quotes from his work. As I read his beautifully crafted and emotionally powerful lines I mourned our loss, while also feeling some pride that the world was giving such recognition to someone who is my compatriot.

Including Heaney, there are four Irish Nobel Laureates in Literature, which is not a bad record for a country with the population of Louisiana. In science we have not done so well though. The Nobel committee awarded only one Nobel science prize to an Irish citizen, to Ernest Walton for splitting the atom in the 1930s. When I was growing up in Ireland I never heard about Walton, but like many Irish people I knew our literature laureates well: Yeats, Shaw, Becket, and Heaney.

The Irish education system does a great job of exposing children to literature. We learn about the deep tradition of poetry in the Irish language, with its special rhyming patterns, its storytelling, and its unique imagery. Our English literature classes teach us the great Irish writers alongside the best from the rest of the English-speaking world. Even in history the recounting of our cycles of repression, rebellion, and defeat are interspersed with uplifting stories of our bards, our poets, and our writers, and the hedge-school masters who defied the Sasanaigh to teach Latin and Greek in secret.

Perhaps the association of the humanities with Irishness was most strongly forged in the early part of the twentieth century. Around that time literature flowered in the Irish Renaissance. Meanwhile political nationalism spread and strengthened, and waged an ultimately successful rebellion that resulted in an independent Ireland. Strength in literature and strength in politics seemed to go together.

Percentage of Working Population with Science and Technology Education in various Eutopean Countries in 2009 (Figure 4.7 of “Science, technology* and innovation in Europe, Eurostat 2011)

Now almost a century has passed. Look at the graph above that shows a measure of how high tech the workforce of each country in Europe is, and note that the third bar from the left is Ireland. In a generation the country has gone straight from a sleepy economy dominated by agriculture to a post-industrial economy dominated by science and technology. Of course, there has been a setback recently where some shortsighted use of financial technology combined with poor economic policy lead to an Irish banking bubble that burst disastrously when the worldwide crisis spewed its contagion across the globe. Still, I think Ireland will come back strongly in the long term, because graphs like this show that its workforce is well matched to the needs of the twenty-first century.

So despite being steeped in a national mythos so tied to writing and storytelling, the reality of Ireland is closer to that of Silicon Valley. Perhaps with this new reality, more Irish people will come to know about Walton, as well as all the Irish scientists, such as Bell, Boyle, Boole, Stokes, Beaufort, Fitzgerald, Hamilton, and Lord Kelvin, whose names live on in science. And hopefully we will add to that list, and maybe even a few more Irish scientists will make that trip to Stockholm.

So, the nation of Ireland as a whole has navigated the two cultures, of the humanities and the sciences. In my own small way I was part of that, for I distinctly remember in my mid teens confronting the decision of what academic direction I should take. I loved writing and remember the pleasure I got from turning ideas into sentences that sounded good. But I also loved Science and remember doing calculus just for fun to work out physics problems.

In the end the larger national level priorities were the decisive factor in how I chose. The government was investing in technology education, and the papers were full of the benefits of getting an engineering degree. Coming out of a working class background on the Northside of Dublin, I did not feel I had the luxury of making anything but the most pragmatic choice, so I choose electronic engineering. It had neither the romance of writing nor the excitement of scientific research, but I would likely get a job when I graduated.

And I do not regret the choice. It has given me a secure and comfortable career, a large part of it working in research labs just like I fantasized as a teenager. And I discovered pretty quickly that I really love programming, that I am one of those lucky people for whom work is never a chore, but something I look forward to every day.

Still, I sometimes wonder about the path not taken. Perhaps I could have been crafting beautiful prose not crafting elegant code, plotting stories not writing specs, observing people who could become characters in my novels not observing them to find usability bugs in my software.

And living here in San Francisco, I see the dance of the two cultures playing out here too. This city, sitting up at the end of a peninsula, has a long tradition steeped in the arts and humanities, while down at the base of the peninsula is Silicon Valley, the ultimate in technology. However in the last decade the valley has extended a pseudopod of yang into the yin of the city, and fertilized a booming high tech industry on the combined talents of designers and coders, of people who understand people and people who understand algorithms.

And this weekend half the city seems to be away in Nevada celebrating that amazing synthesis of art and technology that is Burning Man.

It feels good that the two cultures are coming together in the two places I love, the place where I live and the place where I am from.

immut

image attribution, Frederic-Poirot

What the Internet companies don’t have they can’t give to the NSA

Consider two different types of privacy issues: 1. privacy from governments, which is well regulated by law in both Europe and the United States 2. privacy from corporations, which is well regulated by law in Europe, but not generally in the United States

The latest revelations of how the government accesses user data stored by corporations makes it clear that these two issues are closely related. In particular the vast stores of data that Internet and telecoms companies gather is a mother-lode that is just too tempting for governments to ignore. The more corporations know about us the more the government knows about us.

Of course corporations do not gather this data for the benefit of the government, rather they gather it because it is very valuable to the prevailing Internet business models enabled by advances in machine learning technology. Currently any U.S. corporations that tried to significantly increase the inherent privacy of its users would be at a business disadvantage relative to its competitors.

One way to avoid this race-to-the-bottom of privacy protection would be to have the U.S. companies subject to more stringent privacy protection regulation. By having privacy protection laws applied equally to all companies a single company would no longer be at a competitive disadvantage in protecting user privacy.

Then with our privacy better protected from corporations, our privacy would indirectly be better protected from the government.

How could this come about?

Citizen of the United States could push their government to adopt privacy legislation at least as strong as that of the European Union. The political culture in the U.S. has been to avoid such regulation of private businesses, but maybe now this can be regarded as a way to provide protection indirectly against government intrusion into privacy.

Citizen of European Union countries could push their governments to examine whether existing privacy laws are really being respected by U.S. corporations. For example they might consider whether the current “safe harbor” mechanism is a loophole for avoiding complying with European law. If U.S. companies were forced to behave like European companies, then not only would it enhance the privacy of European citizens, but it would tend to enhance the privacy of people worldwide.

A cynic might say that this will never happen, because it goes against the inherent interests of so many powerful corporations willing to spend a lot of money lobbying legislatures. However there is a techno-utopian undercurrent in Silicon valley that I think is shared by the people running the Internet companies. There is a widespread honest, idealistic belief that their technology can make the world a better place. And that could motivate them to accept policies that are not purely profit maximizing, including those that will help protect user privacy.

In the meantime, if you work for an Internet company, consider as you design your systems whether you can meet the requirements of your business model in a less privacy-intrusive way. Do you really need to store that piece of data? Maybe it can be stored encrypted with the key retained by the user? Maybe it can be stored only as a secure hash? Better still, try innovating on the business model to find ways of making money in new ways that do not involve collecting and storing large amounts of user data. Can you make your user your customer and not your product?

[Originally posted a version of this on Facebook on St. Patrick’s Day, 2013]

Why don't I wear green on St. Patrick's day? Well, when I grew up in Dublin many decades ago, I knew no one who wore green. Instead people would wear either a special badge or perhaps a sprig of shamrock on their lapel. Later when I came to the U.S. I discovered that not only do some Americans actually drink beer dyed green on St. Patrick's day but many of them, even those with tenuous or no links to Ireland, make a point of wearing something green on that day. Like an anthropologist I slowly began to understand some of the peculiarities of American culture and I came to realize that the wearing of the green was part of an odd Hallmark'ized coloring of the American calendar which also called for particular seasonal colors for days around Christmas, St. Valentines' Day, and Halloween. But it would seem odd for me to wear green, to be more Irish in America than I had been in Ireland, so I didn't, and out of stubbornness I still don't. This can be confusing to my Irish-American acquaintances, and I worry that they might consider my pointedly non-green attire as a rebuke of their heritage.

The truth is I did not consider myself Irish when I grew up in Ireland – with the brashness of youth I styled myself a citizen of the world, and considered my moving to London after college as inconsequential as someone growing up in Vermont moving to New York. In fact it was many years later, after my peregrinations had taken me to New Jersey, that it dawned on me that the label that could be applied to me was not my romantic notion of “citizen of the world” but rather that most clichéd label, “an Irish emigrant”.

Over time I have come to be at peace with my Irishness. I have even discovered a liking for traditional Irish music. My politics are still firmly anti-nationalistic, but now their target has broadened: I am as much sickened now by jingoistic nationalism from parts of American society as I was then by anti-Britishness from Irish Republicans. My views on religion are still the same, but now I am as appalled by the science-denying American fundamentalists as I was then by the socially repressive Catholic Church.

But still there is much to celebrate on this day. The most important thing brought to Ireland in that 4th Century cultural transformation associated with the semi-mythical St. Patrick was the gift of written language and of Latin, thus joining us into the great stream of European culture dating from the Greeks. Through ups and downs of the subsequent seventeen centuries we have shaped an Irish culture in which the written word is valued, and we have punched above our weight in world literature.

And in the end, the exuberance of the American celebration of St. Patrick is mostly harmless, and indeed I take it as a good natured recognition and appreciation of the value that the Irish have brought to the American melting pot.

So everyone, hope you had fun today and felt good vibes while wearing green if you chose to do so. Wishing you sláinte as I take a sip from my nightcap, my all time favorite beer – Racer 5 IPA from Sonoma County, California.

http://code.google.com/p/aws-missing-tools/

Amazon AWS has some great monitoring tools for your cloud instances and other parts of your AWS cloud infrastructure. However one notable missing out-of-the box feature is the ability to monitor disk usage of your instances, something crucial for reliable large-scale deployment.

However it turns out there is a way to add custom metrics, including disk usage, that incorporate smoothly into your AWS monitoring dashboard: