Simon Palmer’s blog

June 10, 2013

Deploying a Play 2 App to Amazon EC2 connected to RDS

Filed under: code, Uncategorized — Tags: , — simonpalmer @ 3:04 am

Some more fun and games with deployment of Play to EC2.  Just to set the scene, I have a Play 2 app which was built locally using Java 1.7.  I connect it to an Amazon RDS MySQL instance.  All this works just fine when running from localhost on my machine.

In theory, and according to the Play Documentation, I should be able to

  1. create a standalone Play distributable using play dist
  2. copy that up to my EC2 instance
  3. explode it
  4. run start

So that’s what I tried and it didn’t work.  For a start the standard Linux AMI at Amazon ships with Java 1.6 as the default, so I had to upgrade that (see here for how to do that).

Then I had to install a set of tools to give me access to my EC2 instance.  I like WinSCP which gives me a file browser and removes all the sweat of uploading and editing files on the Linux server, and does it through SFTP so I’m safe.  You can get WinSCP here.

I also needed PuTTY, or more accurately PuTTYgen to translate from the .pem file that EC2 creates for you when you make your security group in which your instances run, to a .ppk file which PuTTY and WinSCP need.  You can get the download from here but I confess I found that page very confusing.  Basically you want the one called “A Windows installer for everything except PuTTYtel“, and get the first one on the list since there seem to be two things with the same label.

Having done that using PuTTYgen is straightforward and you have a .ppk that you can use in WinSCP.  This blog has a nice write-up about that bit, in fact it’s worth a read in general.

Lots of steps to get to the basic point of being able to connect a file browser to my EC2 instance.  One thing to remember is that the DNS entry for your instance will change every time you restart it, so your saved connection in WinSCP will go stale as soon as you restart in EC2.  The way round that is to assign an Elastic IP address, but beware, that costs money.

Having connected I can very simply upload my Play distributable to EC2.  That distributable is just a zip file, so you need to unzip it when it gets there.  It doesn’t really matter where you put it on the server, but it’s simplest to put it in your ec2-user folder as you will have all the permissions you need to do that.

At this point you need a shell on your EC2 instance so you can run commands, including the start script that comes with your dist.  I tend to use the default shell that EC2 offers, but having done all that downloading you could (perhaps should) use PuTTY to do the same thing.  If you are like me (a bit lazy), then click on your instance in the EC2 console and choose “Connect” from the “Actions” menu at the top and then select “Use Java SSH client” and you’ll get a shell.

Now navigate to the folder in which you dropped your zipped dist file and unzip it.  That couldn’t be easier, just type “unzip” and it’ll explode it to the same folder you are currently in.

Now, in theory you should just be able to type ./start and your app should be running.

That didn’t work for me.  I spent many hours while the script was silently timing out in the background googling aimlessly trying to figure out what might be wrong.  It’s worth noting that a /logs folder is created with an application.log in it, which does give some clue.  In my case I was getting the error

2013-06-10 01:43:13,855 - [ERROR] - from com.jolbox.bonecp.hooks.AbstractConnectionHook in main 
Failed to obtain initial connection Sleeping for 0ms and trying again. Attempts left: 0. Exception: Connection timed out

This means “I can’t connect to your database, so I can’t get started”.  I was getting an exception thrown in the SSH client window after a very long timeout, but it actually took me quite a while to spot it.  That had a slightly less cryptic message and told me that I could not connect to my database [default].

That led me down a garden path of believing that it was not trying to connect to the right database and that the application.conf file packed into the dist was wrong or missing.  A few things on stackoverflow (this one and this one and this one and this one) helped, but they basically led me down the wrong path, although the experimentation was a useful learning curve.

So, the real problem was simply that I needed to make sure that my EC2 instance was allowed to talk to my RDS instance.  Turns out you can do this all through AWS.  I had a default security group on my RDS instance and a default security group on EC2, all I had to do was add the EC2 group to the RDS group, which you do through the RDS console by selecting Security Groups and adding an EC2 Security Group, and choosing the one you are using for your EC2 instance.

I re-ran my start script and it happily connected to the database and ran my app.


I still wasn’t done.  When I hit the page from the outside world (having assigned an elastic IP) I got a server connection error from the browser.  The last step is to make sure that the appropriate port is open to the wide world from your EC2 instance.  In the case of a Play app the default port is 9000, although you can change that, but I hadn’t, so I had to add it to the list of open ports in the EC2 security group.  You do that from the EC2 console, select Security Groups, choose the one you are using, select the “Inbound” tab and add a custom TCP rule typing 9000 as the port.

I tried to open this up on standard ports (80 and 8080) and then running the app using “./start -Dhttp.port=8080” but I couldn’t get that to work.  I suspect there are some internals of my Play app which would need to change to reflect the port change, and I was so happy to just see it all working up in the cloud I stopped there.

June 9, 2013

Upgrading to Java 7 on an Amazon Linux AMI

Filed under: code, Uncategorized — Tags: , — simonpalmer @ 2:37 am

OK, so I’m going to deploy a Play 2 app, backed by MySQL, into AWS.  I am using RDS for the MySQL instance, so I’m not expecting any trouble there, but my code does use Java 1.7, as does my Play 2 framework.

If you use the standard AWS AMI for 64 bit Linux (amzn-ami-pv-2013.03.1.x86_64-ebs (ami-05355a6c)) you will get Java 1.6 by default.  To be fair to Amazon, the AMI does include the Java 1.7 openjdk, but it doesn’t use it by default, you have to upgrade, and here’s how you do that after opening an SSH connection to your running instance:

sudo yum install java-1.7.0-openjdk 

All good so far, but then when I typed java -version it was still using 1.6.  So this is the little nugget I am recording in the hope that it might save you an hour of searching.  

To actually switch to using Java 1.7 you need to use the following:

sudo alternatives --config java

This will show both and you just choose 1.7.  Easy when you know how, impossible when you don’t.


March 16, 2011

I love the LinkedIn Map…

Filed under: Uncategorized — simonpalmer @ 1:51 am

You can see mine here.

I’m not convinced it has any value whatever, but I do like it.

October 12, 2010

Toronto – WAKE UP!

Filed under: Uncategorized — simonpalmer @ 5:45 pm

Toronto is a city dominated and ruined in equal measure by cars.

Cars, and their insane drivers, prevent Toronto from being the world-class lakeside city it should by all rights be.  Instead it is a crucible of collisions with a cowering population stuck in the middle on an ugly traffic island.

Here are some thoughts for Torontonians:
1) The average stopping distance for a vehicle traveling 100kph is 80m.  You don’t need to leave 80m between vehicles traveling at 100kph, half that will do.  However, bear in mind that judgment and reaction time alone means you cannot stop within 40m from 100kph.  Nobody leaves 40m.  If you do, at least one car, and probably two, will pull into it.

2) If you travel at 120 while I am traveling at 110kph, and we both have roughly half an hour journey, you will arrive less than 3 minutes before me.  On the other hand the risk you introduce of an accident goes up as the square of the difference between our speed.  That means, if we both travel at 110, we both arrive at the same time and there is minimal risk.  Your extra 10kph buys you a couple of minutes – at the most – at the cost of dramatically increased risk.  In reality you’ll never recover those additional minutes, I’ll pass you again later as the traffic speeds vary during the journey, or we’ll both be held at the same light as we get off the highway 50m apart.  IT’S INSANE.

3) While you may accept the additional risk for 10kph, I do not.  Very few accidents at speed on a highway involve a single car.  You are making a choice for me.

4) More roads = more traffic, not less.  Countless studies prove the conclusions from the original British study about trunk roads (Google Trunk Roads SACTRA).  “An average road improvement, for which traffic growth due to all other factors is forecast correctly, will see an additional [i.e. induced] 10% of base traffic in the short term and 20% in the long term”.  The answer is public transport.  If Toronto is going to be a world class city and not drown under the weight of its own traffic as it is currently doing, you have to improve public transport.  Someone tell the new Mayor when he gets into office.

5) You will never have a usable lakefront until you get rid of the Gardiner Expressway.  It is an ugly monstrosity born out of an outdated and disproven philosophy which prevents Toronto from being a lakefront city.  Think Boston.  Think Big Dig.  Start now.

I want to shake Toronto by the shoulders.  WAKE UP!  Stop driving like morons.  Stop crashing into each other every day on your journey to work.  Stop killing thousands of people on the roads every year.  STOP DRIVING!  Press your public officials to build public transit.  Reclaim your wonderful, beautiful, majestic lakefront.  Be less Birmingham and more Sydney.  Less Shanghai, more Boston.

oh, and before I forget…

6) If you drive a Pontiac Grand Am or Grand Prix, please go to the nearest Police station, surrender your license to an officer and hold your hands out ready to be cuffed.  You are not in a sports car and you do not own the road.

August 30, 2010

Customers as advocates

Filed under: Uncategorized — simonpalmer @ 2:32 pm

Bit of a shameless plug, but I really like the work that’s coming out of our social media team.  GoRecommend started as a tactical integration point so our customers could track, and to some extent guide, where their brands ended up in Facebook following shopper taking a survey about customer experience.

The positioning of a brand is a delicate process which is jealously guarded and heavily invested by companies who care about how they present themselves to the world – which is pretty much every company.  The fact that, driven by loyalty, and perhaps unfettered by it, shoppers can question, criticise, re-position, evangelise, advocate your brand in a selection of forums with enormous networked readership means that the power balance between companies and their customers has changed.  This scares the willies out of most marketing departments and is the sort of discontinuous change that, unless embraced, could spell disaster.

On the other hand, what if you could properly embrace it?  What if you could understand and watch what people are saying?  Done badly it is Orwellian interference, done well it is game-changing connection to your audience and customers.

Enter GoRecommend.  We’re at the friendly fun end of understanding what shoppers are saying about brands.  We provide the infrastructure to make a positive message easy and a negative message contextual and beneficial – after all how can a brand improve unless it listens to its critics?  We’ve extended out beyond Facebook and into Twitter and email and are getting a huge amount of market traction from our customers who are excited and slightly bewildered by the implications of social media.

Come take a look…

August 9, 2010

OnStartups, HubSpot, MadMen etc.

Filed under: Uncategorized — simonpalmer @ 4:41 pm

I read this OnStartups post with interest and tried to figure out my mixed emotions.  Here were my emotions:

Envy… frustration… annoyance… realisation… blog libido

So here I am blogging about it.

I recently joined a great company just outside Toronto (locally referred to as TO) and one of the first issues I face is getting a handle on where from, and how, we can attract talent into our business.  We have a great proposition, a great workplace atmosphere, a great customer list and a great set of backers.  Still, we struggle to attract technical talent from the pool available in the GTA (Greater Toronto Area for those of you outside Canada).

The GTA contains a large number of potential people and some great schools (Waterloo, UofT, Ryerson, McMaster to name but a few) turning out bright grads, docs etc.  It also contains some very different working and living environments, from Waterloo/Kitchener in the West to Markham in the East and Richmond Hill and Thornhill in the North.  And then there’s downtown Toronto, which is a vibrant international city with a diverse cultural and intellectual mix.

Basically there’s something for everyone, and you would think that in an environment like that a company like ours ought to find it easy to attract talent.  But we don’t.  In fact it is devilishly difficult to get a good stream of people into the front of our recruitment pipeline and I have been looking at why.

At this point I need to say that we are in West Mississauga (click for a Google map) which is not in Toronto. The GTA also has the worst traffic of any city I have lived in – by a long margin.  There isn’t really an alternative to a car for a lot of journeys and Toronto also sits on a major Canadian artery running East-West across the country, HWY 401.  Put this all together and if you are travelling into TO with the rush hour traffic, or out again, you can set aside a minimum of an hour in the car each way, and much more if there is any sort of traffic congestion, such as roadworks or accidents.  There are always roadworks and accidents.  I can’t understate how bad the traffic can be.

So, after a number of conversations with local recruiters I was in the process of accepting the obvious, which is that our location is the reason why we can’t mine the rich seam of talent that clearly exists around us.  I was also looking at the feasibility of an office downtown or in the K-W area to drop ourselves into the middle of the melting pot.  This is risky and expensive if it doesn’t work, so I’m not apologizing for thinking carefully about it.

As I did so I realised I was looking at location in the wrong way, it isn’t the cause any more than salary, package, working environment or technology choices are the cause.  In fact all location does is place another constraint, just like all those others, on the sorts of people who may be interested in working for us as a business.

Then the OnStartups blog post emerged and my cycle of emotions kicked in and this is where I got to.

I reject the notion that all good tech companies must, by definition, be staffed by young people, and that you have to create a workplace which panders to their very specific social needs.  In fact, those exact same people grow into mature adults with kids and a different set of needs.  As that happens none of their talent goes away – in fact it is augmented with experience and broader skills which make them more valuable, not less.  Why do they have to go from a loft downtown to a scrapheap in the suburbs?

So I wondered, what does the adult version of that set of (non-)policies in the OnStartups post look like?  I ask partly because this is a tier of the workforce that these sorts of businesses are clearly ignoring and partly because I am a crotchety old 40-something who resents the notion that all the talent exists below a tree-line which ends at 29, and I secretly harbor the notion that they are actually a bunch of  lazy and self-possessed children who have had very easy lives and think that the world owes them something.  Plus none of them have yet proven much.

If we can’t attract the kids to Mississauga because they are drinking beer and playing foosball downtown, then what about them in 5 years time when they have succumb to their girl/boy/friends and have moved out to a house with a minivan, a dog, a school round the corner, a back yard and 2 years of 3 hours sleep a night?  They are the same talented people but they don’t work as long and they care about a more structured set of policies on vacation, working hours, etc.

They may also have been dumped unceremoniously out of because the fancy office couldn’t be paid for out of revenues and their financiers called in their warrants when the hockey sticks turned out to be wooden spoons.  So they are more realistic and perhaps less productive – but they carry all the skills and talent, and they can’t all be managers.

I wonder what an ideal workplace and set of policies looks like for them…. (being provocative)… I want to create a “post-juvenile culture”

  1. Vacation policy = closed for Christmas
    …and 2 weeks in August
  2. We care which 37.5 hours a week you work
    …because we think you should spend quality time with your kids, because they’ll be happier and you’ll be happier, more loyal and will work harder while you are here – plus experience tells us that you’ll probably do better if you are mostly here – which aligns with the Agile manifesto which we espouse in reality, not in theory.  And we also know that a motivated, but otherwise committed, person like you will do whatever is necessary to make yourself, and us, a success.
  3. Extreme filtering
    If you want to know what’s going on in the heads of management come and ask.  We don’t want to blur your very busy, and slightly shortened, day with a load of irrelevant information for decisions in which, ultimately, you are unlikely to have much say.  It’s not because we are trying to hide anything, and where legally permissible we’ll fully disclose.  Instead it’s because there is always a strata of information which matches your role and makes you more effective, and a whole load of other stuff which is largely irrelevant to you – and we understand that and see it as part of our role to protect you from it
  4. Some doors policy
    We acknowledge that there are moments when you deserve some privacy to talk about things which are important to you.  As management we face this every day and don’t want you to feel like you have to air your dirty laundry with us as we sit between two developers.
  5. Free breakfast
    You probably squeaked out of the house with no time to grab a bite, come and have it on us at your desk
  6. Free beer
    Once a month at a bar/restaurant.  We’ll help you organize a babysitter and either a DD or a taxi home.  Please don’t drink in the office and drive home.
  7. Partnership with a local crèche and daycare
    We understand that the biggest challenge you face with young kids is what to do with them while you maintain some momentum in your career and how you deal with finding good quality care that you can both trust and afford.  You may not have a fancy office and a really great healthcare plan, but we’ll help with childcare costs – especially beyond the hours that the providers normally offer.  That means you don’t have to feel like you have to rush away at 2.30pm to drive the 45 minutes to daycare to pick up the kids while you do two conference calls on your earpiece in the car and wonder how long it’ll be before it becomes a career limiting issue.
  8. Social media policy = whatever
    Really, we care as little as you do
  9. Dress code = be respectful
    We are a professional organization and you are a professional person, your attire should reflect that.  We also have customers in the building on occasions with whom you will have worked very hard to establish the professional credentials of yourselves and our business.  We know you would not jeopardize all that hard work by dressing inappropriately.

That’s as far as I got before I realised that it was compelling enough to make a very nice workplace proposition in its own right.  I’m actively pressing this agenda with the people we think we would like to attract.  I’m also pursuing it internally to see how practical we think it would be for us as a business – we have to balance this agenda against our commercial business objectives – which are always missing from this sort of discussion.  I like it because it doesn’t necessarily mean we have to get a loft downtown to attract the talent – some of it is probably living within 20 minutes in the car, we just have to find it.

January 7, 2010

Clever little fella

Filed under: Uncategorized — simonpalmer @ 2:52 pm

I was talking to Jack this morning and he was repeating back to me what I said. Then he suddenly started changing the odd word so that it was funny. This is a nice little game that he and I play quite often and I’m sure is quite common between parents and kids. It causes him to belly laugh when he finds a word which changes the meaning of the sentence and makes it funny.

If you think about it a sense of humour is a very sophisticated layer of communication which sits above mastering the mechanics of speaking, the detail of language and the meaning of words.

I was amazed. 6 months ago he was in nappies and barely able to talk. Now we are playing word games and he is deciding what is funny. That sort of transformation must belie a huge change inside his head.

I think I subscribe to the Noam Chomsky, Stephen Pinker view of the world that we have an innate ability to master language, and I suspect that the real key to it all is the development of memory. If you consider everything that is necessary to be able to communicate, vocabulary, syntax, grammar etc. it is all predicated on being able to remember things. I suspect that the transformation I have witnessed in Jack is the development of his memory.

He suddenly has an amazing vocabulary. Yesterday as he was struggling with unzipping his coat and he said quite clearly “absolutely ridiculous”. Later that day he was sitting on the loo and he looked at me and said “concentrate”. These are very sophisticated ideas and there is a complex context necessary for them to be applied appropriately, which they were.

Memory development is supported by the other amazing things he has started to do which is remember directions and locations. If we drive out from our home towards the North of London he can now tell whether we are going to the shopping mall, the chinese grocery store or Nanny’s house by the route we take right at the last moment.

So I’m obviously a proud dad – he’s only 2 1/2 – but I am also an amazed observer of the rate of development of the human brain.

December 20, 2009

Google Wave. Ho Hum.

Filed under: Uncategorized — simonpalmer @ 9:54 pm

My wave account is now a month old. I have about 20 fellow waving friends. We don’t use it. As far as I can tell they don’t use it either. In fact, who does use it, other than the development team who built it? I never remember to check it and it doesn’t flag activity to me via email, which I’m sure is deliberate and would be tautological in the eyes of the wave creators.

I was talking to a software friend based in California at the start of this week. She has people developing software in pretty much all the major centres in the world. I asked whether she either used, or intended to use Google Wave and she didn’t. Not because it may not be the best thing since sliced bread, but primarily because she already had processes and systems in place which worked perfectly well and which were embedded into her teams.

Of course that presumes that she had concluded what it was that Google Wave does, and when I asked she waffled a bit but basically said “collaboration”. That’s a big space, but it seems to be where most people place Google Wave when thinking about mapping it into their world.

It is generally a big ask to have all your teams switch working practices, especially core traits like how they communicate with peers and document that communication. Contemplating such a switch generally requires a backdrop of discontinuous change or profound dysfunction, to make it worthwhile, and I guess Google may claim ground as agents of discontinuous change with Wave. I’m not convinced though.

I’ll keep my account of course, but until I find a need it remains a Faberge egg. A fascinating, if useless, enigma.

November 12, 2009

Google Wave

Filed under: Uncategorized — simonpalmer @ 4:59 pm

So I got my Google Wave invitation today (from the same person who invited me to GMail many moons ago – mucho gracias fooch) and I logged in for the first time today. Here are my first observations…

Ouch, scrollbars don’t work properly in Firefox 3.0.15, pretty fundamental and visible feature and surprised they don’t have a commodity widget like that working. Even more surprisingly, when I try to run it in IE to see if it works there I’m asked to install the Chrome Frame plug-in. Small things perhaps, but shop-window problems which make it feel pre-beta.

Layout is simple and clean and in the Google mould. Bits of my gmail profile, picture and contacts for instance, have made it across. It is obvious how to create a new Wave and add people to it and I have started a couple with the contacts I have which appear to have wave accounts. I have a address, which mirrors my gmail address, so I try and send an email to it to see what happens. So far nothing has happened. It didn’t bounce and nothing arrived.

Weirdly I was expecting to see my gmail inbox in here. I don’t know why really, I was just expecting it. And that leads me to the obvious question. If this isn’t email, then what is it? What can I wave that I can’t email?

I have a couple of waves already set up, several welcome ones from Dr Wave (c’mon, are we really patients? or does it need a PhD to figure it out) containing video clip tutorials and explanations. After watching a few the idea seems to be a cross between multi-party email and chat, with sharing of documents, pictures etc. and the whole in a chronological sequence of evolution of the interaction. You can (apparently, and eventually) use it for collaboration on documents, creating events and the like. I’m beginning to get the picture.

Trying to map it into the MS Office world, I think it sits somewhere between Sharepoint, BPM, MSN, Word and Outlook. It is certainly a different take. I think my main issue at the moment is that the benefit is not clearly articulated. How is it better than working the old way, because let’s face it, most of us do most of these things today, and there is overlap between them already?

It may make corporate buyers look again before investing in collaboration infrastructure like Sharepoint, although it is the smaller players in that market, and the open source projects that are likely to feel the ripples from the wave, not Microsoft.

So those are my first immediate impressions and I’m going to hedge my bets at this point. There may yet be something very illuminating underneath it which I am yet to see. Maybe, as we all take to smaller devices, there is some new way of working which wave is anticipating.

However, it has more than a little of the segway about it, and unfortunately a bit of the Sinclair C5 too. Innovative and clever? Undoubtedly. Challenging to established norms? Certainly. Paradigm-shifting? Hmm, maybe not. I have the feeling that we are all still going to be emailing in 5 years time. Google Wave will almost certainly find a niche in some CaliTechy communities, as has the segway, but I’m still walking and driving rather than segwaying, and I can’t see my wavebox displacing my inbox. Yet at least.

August 24, 2009

Office 2007 Ribbon

Filed under: Uncategorized — Tags: — simonpalmer @ 11:55 am

I know I am a bit behind the rest of the world with this, but I just switched to Office 2007 (still on XP, I have seen nothing about Vista that I like or need). The really big change is the ribbon, which was a bold UI step given that you have to completely re-learn how to use the Office tools.

I’m not sure whether I like the ribbon or not, but I can see how I will get used to it. Truth be known, I can’t really claim to like the “classic” office menus either, it is more that I am familiar with them. If the ribbon had appeared before classic menus then the switch in the other direction would probably feel worse, so I don’t want to bash the ribbon – I just need to get used to it and I’m sure I will.

However, what I really don’t like is the amount of space that the ribbon takes up. I wouldn’t mind so much, but you can’t customise that element of it, so you are stuck with a good portion of the screen taken up by a bunch of controls you probably will never use.

I took the time to read a bit about what other people say about the ribbons as I was looking for ways to customise it. I think most of the reasoning was somewhat specious and self-serving since it ignores the fact that the real change is in the degree to which you can customise the ribbon bar and recover the real estate that it takes up. Pretty much all the hard-core Office users I know, particularly Excel users, have customised their toolbars to maximise real-estate and remove clutter, so, sorry, but I don’t buy the argument.

I just switched to a netbook with a 10.1″ screen and I really care about screen real estate. I was rueing the fact that I lost all the control I loked with the old classic toolbar and beginning to curse the ribbon when all was redeemed by mistakenly hitting CTRL-F1.

What CTRL-F1 does is shrink the ribbon so it only shows when you press on the menu item. Maybe I’m just attuned to this because I have a small screen, but it looks like a much better default setting to me…

Older Posts »

Blog at