Hashbang.ca

Google has been working for about 2 years on a book about Site Reliability Engineering, the discipline and organization that keeps Google’s large-scale systems running smoothly. “Site Reliability Engineering” was finally published last week. It spans some 500 pages, and offers a rare inside glimpse into how Google actually works. The authors are remarkably open, naming technologies and projects, and explaining how systems work. There may not be source code, but there’s lots here that can be implemented outside Google. That makes it a great read for startups expecting to scale and small-to-medium tech companies that want to up their reliability game.

At work, we encountered a sudden and precipitous performance regression on one particular page of a legacy application. It’s a Perl web application, running under mod_perl, using ModPerl::RegistryLoader to compile scripts at server startup, and Apache::DBI to provide persistent database connections.

Our users suddenly began complaining about one particular page being “three times slower than normal.” Later examination of the Apache logs showed a 20x(!!) slowdown.

Investigating this performance problem was interesting because we didn’t have good access to required data, and our technology choices slowed us down or completely prevented us from collecting it. Although we solved the mystery, the experience had several important lessons.

CloudAtCost.com provides low-cost virtual machines hosted in Canada. panel.cloudatcost.com is the management interface, where customers can install OS images, access a console, etc.

A cross-site request forgery (CSRF) vulnerability was discovered in this web application. If a customer could be tricked into visiting a crafted URL while logged in, an attacker could change the victim’s password, gaining access to the management interface.

This Thursday, I gave a talk at AtlSecCon 2014. The weather threw a wrench in the organizers’ plans, but they managed to pull off a solid conference. Unfortunately, the talks weren’t recorded this year. The slides are posted on SpeakerDeck, and are embedded below the fold.

I also reprised this talk at NSLUG, and recorded audio, now posted on SoundCloud, and also embedded below the fold.

Finally: late last year, I wrote 3 posts exploring Canada’s computer crime laws (1, 2, 3) which were initial versions of work that eventually became two papers I submitted this semester for a directed studies course. If you were interested in those posts, I’ve embedded the final PDFs below. The talk is a condensed version of that work.

Heartbleed is a critical vulnerability in OpenSSL revealed yesterday. I’m not sure it could be more serious: it allows an attacker to connect to your server and use the TLS heartbeat extension to obtain 64k of server memory (and do it again to get another 64k and again and…) – while leaving no traces in logs. That server memory might include primary key material (private keys), secondary key material (usernames and passwords), and collateral (memory addresses, canaries used to detect overflow, etc)

I spent about 3 months interviewing with a number of companies in Canada and the US, and I was lucky enough that list included an interview with Google’s Site Reliability Engineering team. I went down to the Mountain View campus again in December for an on-site interview. Although the process was daunting, I made a good enough impression that they’ve invited me to join the SRE team in Mountain View when I graduate this May as a Systems Engineer.

If you encrypt your Android device, the standard over-the-air (OTA) upgrades don’t work, because /sdcard can’t be mounted in recovery.

Since the exceptions in copyright law for encryption and security research don’t apply if you’re doing anything criminal, I next looked at the Criminal Code [PDF].

Since the exceptions in copyright law for encryption and security research don’t apply if you’re doing anything criminal, I next looked at the Criminal Code [PDF].

As someone with an interest in technology, security, and the legal issues surrounding them, I often watch relevant legal cases with interest. Typically, those cases come from the United States. The CFAA has been in the news frequently of late, and not always in a good light. I was pleased to see Zoe Lofgren’s proposed changes, which try to make the law less draconian.

This is typical for Canada – we often see more about American news on topics like this than Canadian. I realized that I really didn’t know what the law in Canada said about so-called computer crimes, although I’ve often wondered. A while back, I took an afternoon to do some reading. I was not happy when that afternoon ended. This is part one of a three-part series on what I found.