Coding Exercise Silliness

Reading Time: 4 minutes

It happens quite often that we assign a simple programming task for one reason or another. One thing we find is that our victims often look for the “right” answer when in reality there isn’t one. Sure some answers are better than others but even that can be subjective.

So whilst a group were busy scribbling away I thought I’d demonstrate the point by doing the same problem… in as many different ways as I could think of.

The Task

The task we set them was;

Given the string “We Wish You A Merry Xmas”, output a list of the unique letters.

So I put the string into a string variable called “leString” and got to work. My favourite implementation is this;

Console.WriteLine(String.Concat(leString.ToLower()
    .Where(x => char.IsLetter(x)).Distinct()));

If you understand a little about LINQ, particularly the extension method syntax, then this is about as clear and concise as you can get. It’s possibly not the fastest way of doing it, but speed of execution is not the only concern.

I then tried to guess how I thought a junior programmer would approach it. The result is this.

HashSet<char> alreadyThere = new HashSet<char>();
foreach (char c in leString)
{
    char lc = char.ToLower(c);
    if (char.IsLetter(c) && !alreadyThere.Contains(lc))
    {
        alreadyThere.Add(lc);
        Console.Write(lc);
    }
}
Console.WriteLine();

This is perfectly valid and indeed relatively quick. I don’t think it’s quite as obvious what it’s doing as using .Distinct(), but if you’re not a LINQ Ninja then it’s OK.

I was right here in that it’s this kind of approach that most of them went for, although we saw arrays, lists and dictionaries used to store the list of chars and nobody realised that you could output the chars as you went so the real results looked more like this;

List<char> alreadyThere = new List<char>();
foreach (char c in leString)
{
    char lc = char.ToLower(c);
    if (char.IsLetter(c) && !alreadyThere.Contains(lc))
        alreadyThere.Add(lc);
}
foreach (char c in alreadyThere)
    Console.Write(c);
Console.WriteLine();

There’s a fundamentally different approach that we can take here. Essentially we want to remove duplicates. If we sort the string then all the duplicates will be next to each other. So all we have to do is iterate through the sorted string and avoid outputting the same char twice.
This is pretty easy to implement.

char lastchar = (char)0;
foreach (char c in leString.ToLower()
    .Where(x => char.IsLetter(x)).OrderBy(n => n))
{
    if (lastchar != c)
        Console.Write(c);
    lastchar = c;
}
Console.WriteLine();

It’s a bit of a nasty mix of LINQ and more traditional iteration but I still think it’s pretty clear. We take the string, eliminate anything that isn’t a letter and then sort the result.
We then iterate over the result. If this iteration’s char is different from the last iteration’s char then we output it. If it’s the same, we don’t.

You can also implement this using a for loop, but you need to make the sorted result into an array or list so that you can get a count. This is all rather convoluted so the foreach is preferable in my opinion.

You can implement it in LINQ too. The problem is how to assign “lastChar” after it’s tested. I’ve used a nasty little hack here – in C# an assignment actually has a return type that is the value of the assignment. So lastchar = p actually returns the value of p as well as assigning it to lastchar – hence the final .Select.

char lastchar = (char)0;
Console.WriteLine(String.Concat(leString.ToLower()
    .OrderBy(n => n)
    .Where(x => char.IsLetter(x) && lastchar != x)
    .Select(p => lastchar = p)));

This isn’t good code, it’s not really clear what it’s doing.

Here’s another slightly leftfield LINQ based approach. In C# the ++ actually returns a value, if it’s used postfix (i.e. i++) the value returned is what i was before it was incremented. If it’s used prefix (i.e. ++i) then it’s the value after it’s incremented.
We can use this little gem to index into a counter array…

int[] hits = new int[26];
Console.WriteLine(String.Concat(leString.ToLower()
    .Where(x => char.IsLetter(x) && 0 == hits[(byte)x - (byte)'a']++)));

This assumes that the string is 7 bit ASCII. It goes horribly wrong if it’s unicode. This should be a pretty quick implementation though.

Here’s another 1 liner (if you ignore the declaration of a HashSet). This works in a similar way to the previous example but is unicode safe. A HashSet is a collection of unique values, duplicates are not allowed. The HashSet.Add() method returns false if the entity you’re trying to add is already in the collection. So we can use that in the same way we used the array and the ++ operator above…

HashSet<char> alreadyThere = new HashSet<char>();
Console.WriteLine(String.Concat(leString.ToLower()
    .Where(x => char.IsLetter(x) && alreadyThere.Add(x))));

If you’re a fully paid up, card carrying lunatic you can of course do it parallel…

ConcurrentDictionary<char, int> alreadyThereDict = new ConcurrentDictionary<char, int>();
Console.WriteLine(String.Concat(leString.AsParallel()
    .Select(c=>char.ToLower(c))
    .Where(x => char.IsLetter(x) && alreadyThereDict.TryAdd(x,1))));

Rather irritatingly there’s no such thing as a ConcurrentHashSet so we have to use ConcurrentDictionary but the approach is identical.

Another different approach is this.

Console.WriteLine(String.Concat(leString.ToLower()
    .Intersect("abcdefghijklmnopqrstuvwxyz")));

The result of intersecting a string with a set that contains all possible characters must be a set containing the characters that are in the string. Clearly this isn’t unicode safe either, although it’s better than the array indexing example because this won’t blow up, it just won’t work with unicode.
The advantage of this approach however is that it very clearly defines what the dictionary of permissible character is. So if you wanted to define a slightly different dictionary you could – say consonants for instance.

Finally we return to the original method, but we do it slightly differently. Most of the functionality we think of as LINQ is provided through extension methods defined in the “Enumerable” class.
You don’t need to call an extension method as an extension method at all, you can in fact call it as a static method of its defining class. Thus we can perform the entire operation in a functional manner.

Console.WriteLine(
  String.Concat(
    Enumerable.Distinct(
      Enumerable.Where(
        Enumerable.Select(leString, char.ToLower), 
      char.IsLetter))));

If you’re not familiar with this style, basically you read it from the middle out. So we take “leString” and call Enumerable.Select on it, passing char.ToLower as the predicate for selection. The result of this operation is then used as the first parameter to the call to Enumerable.Where. The other parameter to the Where call is the predicate char.IsLetter. The result of this operation is then used as the first parameter to a call to Enumerable.Distinct and the result of that goes to String.Concat.
As with many things programming it’s horses for courses and this is not the right horse. It’s useful however to know that it can be done like this.

Over to you…

Naturally this isn’t an exhaustive list – it’s just the ones I happened to think of, if you can think of another way let me know!

That Sinking Feeling…

Reading Time: 7 minutes

Beer!“WE MUST HAVE THIS FIXED ASAP!” yells the customer’s head of ICT. The application stopped working some time last week and your support department haven’t managed to fix it yet.

“OK, OK,” you say, knowing that you will come to regret these words, “I’ll VPN in first thing tomorrow and sort it out. Will you send me the connection details.”

“I’LL SEND THE CONNECTION DETAILS IMMEDIATELY I GET OFF THE PHONE AND YOU CAN CALL WHEN YOU WANT TO CONNECT”, replies their head of ICT, because heads of ICT ALWAYS SPEAK IN CAPITALS.

08:00 the next day, no email. Send an email to the customer’s head of ICT reminding them that you need the VPN details.
No response. OK, fair enough, a lot of people don’t get in ’til 9am.

09:30 still no response. Email a bunch of other guys at the customer’s site asking if anyone knows the VPN details and is actually authorized to give them to you. Get a response saying that only the head of ICT is and he’s in a meeting ’til 10:30.

Wait ’til 10:45, no response. Try to find a phone number for the customer’s head of ICT. Apparently he doesn’t have a phone. Email him again. 11:15 get a rather terse email containing the connection details and the username and password in plain text.

Try them, they don’t work. No phone number in provided in the email. Try to find a phone number of anyone in the organisation who might actually be able to help. Find an old mobile number that now belongs to a photocopier salesperson who can’t really talk because she’s on the motorway, but she could do you a really good deal on a pre-owned Canon IRC3380i if you call back later.

Google the customer to try to find a switchboard number. End up in a multilevel call menu system where the only human option is to speak to a “customer relationship manger” about your account. Try that on the offchance. He thinks you’re from their ICT department and has no idea what you’re talking about.

11:30 Remember that you may have stored some old details for the customer’s other site, log in to the password vault. Find some that are 4 years old. Try them anyway, are genuinely surprised when they work. Make a mental note to inform the customer that they seriously need to do a security review.

Connect to the server the head of ICT said was the right one. Credentials don’t work. Neither do the stored ones. Email the head of ICT and everyone else you can think of who might possibly know how to log on to the server. Get no response.

12:00 Remember somewhere in the back of your mind that you once configured SQL Server authentication for one of their systems and there’s just a chance that they re-used the username and password from a domain account.

Log into the secure store and search the archived configs. Find a likely candidate and try the connectionstring details.

12:15 Scratch the head of ICT’s name into the side of a giant security rocket and direct it at his arse when said credentials actually work.

Spend half an hour setting up the diagnostics. Scratch head as to why nothing’s happening.

13:00 deduce that it’s the wrong server. Email the head of ICT and go for lunch.

13:30 return to an email from the head of ICT who is now working from home and doesn’t have the details to hand but has emailed an unnamed member of his team asking them to send them on.

Log back in to the wrong server. Expand the network section and have a look at the machines listed there to see if any of them look likely. No, but it does spawn an idea. Look back at the connectionstring from earlier and note that the Data Source is a name that ends -SQL try connecting to the same name with -GW on the end.
No dice. Try various combinations.

13:47 Eventually try -SVR4 and it works. Try the credentials from the connectionstring, they don’t work. Try the original details the head of ICT sent in the email, they work, but Remote Desktop is not enabled for that user. Try VNC with the same credentials, just in case. Nope.

Email the one person who responded earlier, carefully include a request for a phone number.

14:15 Get a response that contains no phone number but which politely explains that although the person can’t make the change Jack will when he gets back from lunch at 14:30.

14:45 Try logging in again. Still not enabled. Take 6 guesses at what Jack’s email address might be and email them all.

14:53 Field a call from the Customer’s Operations Director who is extremely angry that the system is still not working and your bungling incompetence at not being able to sort out your own software. Explain that there are some technical problems with the VPN connection that you’re trying to resolve right now. Try to persuade him that half hourly updates are not going to help anything and in fact are only going to slow things down.

15:07 Get an email from Jack confirming that he’s enabled the user for Remote Desktop. Log in to the server. Go to the application directory under Program Files and check the version numbers. Not only are they wrong they’re actually a mixture of the past 3 releases. Take a backup and install the latest versions.

Try to start the service. No dice. Scratch head. Just double-check the service executable location. Find it’s actually in “C:\Temp\Barry’s Memory Stick\From Old Server”. Start to get rather concerned that the customer’s assurances that “No we haven’t changed anything” omitted one small detail – the fact that they installed the app on a totally different server.

Look in the event log at the error that’s actually being reported – it transpires that none of the well documented pre-requisites for installing your application have actually been installed. Attempt to download them only to find that the server can’t connect to the Internet.

Start downloading them locally instead and pushing them one-by-one to the remote server.

15:35 Field another call from the Customer’s Operations Director who is extremely angry that he hasn’t got the half-hourly update that you talked him out of and that the system still isn’t working. Explain to him that escalating this won’t help because you are the most senior person that could possibly work on this and it’s your number 1 priority, the only thing it will do is create more admin load for everyone including him. Pretend to try transferring the call to your own CTO’s phone, pretend she’s not answering.

15:55 Ring your own CTO to let her know the situation and to expect an angry call imminently.

16:03 Final prerequisite installed; the service now starts, but it can’t connect to the database. Check the connection string. Note that it’s trying to connect to your own testing server – this is clearly the default config that someone has carelessly copied over the site config. Look at the other copy of the software and find a comment in the config that say’s it’s from Barry’s test system. There appear to be no backups of any version of the config.
Search the entire filesystem for anything that might be a past version of a valid config whilst taking a few random guesses at what the SQL Server might be called.
Email Jack asking if he knows what the configuration should be or at least where the old server is.

16:19 Answer a call from your own CTO who would just like to confirm that you really are doing everything you can. She tells you not to worry about “that email”.

16:21 Receive email from your CTO to their Operations Director into which you’re Bcc’d assuring him that his problem is “our top priority” and that disciplinary proceedings have been started against the employee he spoke to earlier for his “bad attitude” and that the situation is now being dealt with by “someone more senior”.

16:27 Get a call from a random junior techie at the customer’s site who has no idea what you’re talking about but has been told to sort it out because the Operations Director is very angry and wants something done. Note the phone number he’s called from. He promises to find the information you need, ensure you get his name.

16:47 No emails or phone calls, so call the number back. The person answering the phone is not the person you spoke to earlier and doesn’t know who he is, but thinks he may be “that new guy from ICT who was here earlier”. Try to get anyone’s actual phone number out of the person you’re speaking to who says he’d gladly give them out if he actually knew any of them. He transfers you to the Helpdesk, however. The guy you spoke to isn’t in the office and it’s a Thursday so Jack has gone to pick his kids up, but Barry’s about if you wanted to talk to him.

16:52 Barry sheepishly explains that it is actually the same server but that it’s got an entirely new RAID array because nobody noticed when the first disk in the previous array died, but they noticed really rather a lot when the second one did and they were hoping that they could get it fixed before anyone – especially the head of ICT – noticed. They did find a backup, but after the last security scare the Operations Director hired an external security consultant to come in and do an audit and he said that this server should be firewalled from the main network and nobody realised that this would mean that the backups stopped working and nobody noticed that either. So the backup was from just after the OS was installed and they tried to work out how the app should be configured.

Barry is fairly sure which server the database is on, but he doesn’t know which actual database because they’re all looked after by Dave who’s the DBA and he’s on long term sick leave with a major case of stress.

16:57 Talk Barry through SQL Server Management studio and find a database that looks like a good candidate and has recent entries.

16:59 Log on to the remote server and enter the connection string into the config. Start the Services snap-in and try to click the start button. No reaction. A few seconds later Remote Desktop says the connection is lost and it’s trying to reconnect. Realise that it’s exactly 17:00 and vaguely remember something about “office hours only” in the support agreement. Check the VPN, it’s down and wont reconnect.

17:03 Email your CTO with a progress update, casually scan Jobserve.

17:10 attempt to shut down laptop to find 12 Windows Updates waiting.

17:13 ignore the incoming call from their Operations Director, leave the laptop where it is and go to the pub.

At The End of the Day

Fortunately this wasn’t a real day, rather it’s an amalgamation of things that have genuinely happened to me either trying to work on a remote site or in some cases when I’ve actually been there.

My favourite example of a customer having “not changed anything” is in here but it’s somewhat watered down from the reality. They’d had a security audit and much like the story had added a firewall between two parts of their network. They’d also deleted a bunch of users from the database where they didn’t know what those users were for.

What they’d done was to revoke the server’s access to its database and put in place a firewall that not only prevented the clients from accessing the server but stopped the server accessing the external web service that it needed.

Apparently they definitely hadn’t changed anything and the application suddenly stopped working so therefore it couldn’t have been their fault.

ISAM Insanity

Reading Time: 3 minutes

DSC01658SQL Servers are great general purpose tools. They’re popular because they perform OK at just about everything you could want to do with data.

If you want to do something specialised however then there are almost always better techniques. Recently I needed to get medium sized sets of records identified by key, dispersed at random over an essentially read-only data set.

MS SQL Server is OK at this, but there are a bunch of products out there that are far better at this task. At a basic level though it’s not actually a difficult thing to do. It’s just a key-value lookup after all. So I wondered if in a few lines of code I could out-perform SQL Server for this one specific task.

Essentially all I did was to dig up the ISAM file. All that really means is that the main data is serialized to a file. All we store in memory is the ID (that we search for) and it’s location within the file. I then implemented a balanced b-tree to store the relationship between the ID and the file location. You could use a Dictionary<Key,Value> for the latter, it would be a bit slower but it would work. You really don’t need to write a lot of code to implement this.

So in ISAM vs. SQL Server I have on my side;

  • A simple, really fast search algorithm
  • No need for IPC.
  • No need to parse the query
  • No need to marshal writes, manage locks or indeed worry about concurrency at all

SQL Server has on its side;

  • Huge expertise and experience in optimising queries and indexes
  • Super-advanced caching meaning it doesn’t face the same I/O problems

Before the days of the SSD I’d say attempting this was pretty insane. The main problem with the performance of a mechanical disk is the head seek time. You can get data to and from the disk pretty fast provided that the head is in the right place. Moving the head however is massively slow in comparison. So if you have a large file and you need to move the head several times then you’re going to lose a lot of time against a system that has an effective cache strategy.
Half the battle with an ISAM file was to get the records that were going to be read together physically close to each other in the file so that the heads didn’t move much and the likelihood of a disk cache hit was much higher. It’s this kind of management where complex RDBMSs will always have the upper hand.

However SSDs have no head seek time. It doesn’t matter if you need to read from many locations. This rather levels the playing field.
Interestingly I did notice a small speed improvement when I implemented a parallel ID lookup and sorted the results so the FileStream read in sequence from first in the file to last. I’m guessing that this is down to the read-ahead cache (which is still faster than the real storage media).

So did I make it? Can a simple technique from yesteryear embarrass today’s latest and greatest? I wrote a little console app to test…

ISAM Performance Failure
ISAM Performance Failure

Not bad, SQL Server is 15% faster which is a pretty reasonable result. After all I couldn’t really expect to beat SQL Server with just a few lines of .NET, right?
Yeah, I’m not beaten that easily. So I cracked open Visual Studio’s Performance Explorer and found an obvious bottleneck. It turns out that .NET’s BinaryFormatter is dog slow. I tossed it out and replaced it with ProtoBuf-Net.

ISAM Triumphs!
ISAM Triumphs!

Oh yes, that’s much better. This is pretty consistent too. I was using SQL Server localdb which actually seems to be a reasonable analogue. It might be interesting to see it against an enterprise level installation. I suspect that the additional network latency will compensate for the additional performance at it will perform about the same.

So in other words, yes if you have a very specific task and you know what you’re doing then you can outperform the big boys.

So I suppose I should publish the source – but right now it’s horrible. Its only purpose was to prove that I could do it. I’m intending to tidy it up and post it in a different article.

Oracle: Stop Trying To Trick Me With Ask Toolbar

Reading Time: 2 minutes
Ask Toolbar: Nothing to do With Java
Ask Toolbar: Nothing to do With Java

I’m calling Oracle out on this.

I spent the first 5 years of my career working with Oracle and I used to like them. Now though the only regular interaction I have with them is unticking the “would you like to install the Ask toolbar?” box every time there’s a Java update.

It does not give a good impression. Installing software isn’t something that’s particularly well understood by the man on the Clapham Omnibus. There is a certain fear that if you don’t do what you’re told then the software might not work properly. So most people just accept the defaults on the basis that this should work.
There is also the factor that software updates often ping onto your screen when you’re in the middle of something else. So you want to get it out of the way quickly. Again this tends to make people just click the defaults.

If you do that here you end up installing a piece of extra software that has nothing really to do with Java other than it being owned by Oracle. What’s more it’s something you almost certainly don’t want.

I have no problem with vendors advertising their other products in an install sequence and I don’t have any problem actually with them offering to install it.

There are two things that I do consider bad practice.

  1. Failing to clearly distinguish that you would be installing something other than (or as well as) the product you initially intended.
  2. Installing the software by default. The default should be not to.

Oracle isn’t particularly bad on the first one, one could say that the dialog looks like you’re accepting the licence agreement for Java but actually it’s reasonably clear. Oracle are a little marginal here for me. There are far, far worse offenders out there.

The second however is a straight red card. No way should an installer by default install something that the user didn’t ask for.

Please tidy your act up, Oracle.

Tarmac and Tempe

Reading Time: 6 minutes

It could happen to anyone, you’re sat in a great little restaurant in Indonesia and someone suggests that you do a cycle tour of local home industry.

Naturally what I should have said is, “You must be joking! This is Java’s second largest city, the roads have at least 6 times the amount of traffic they can cope with, lane discipline is non-existent, thousands of motorbikes zoom in all over the place at random, I’ve not found a single working seatbelt since I arrived and I’m pretty sure that every single one of the myriad of trucks is on Euro NCAP‘s top 10 blacklist of cycling death-traps”.

 Jl. Prawirotaman and The Grand Rosela Hotel
Jl. Prawirotaman and The Grand Rosela Hotel

That’s what I should have said. I have however been to India and I still have the image of a very brightly coloured but nonetheless very heavily built truck burnt on my retinas from when my tuk-tuk driver went through the central reservation into the fast lane of the opposite carriageway. Apparently it was “less busy”, apart from the enormous pile of steel hurtling towards us at 50mph, that is.
I have no idea how we got back on the right side of the road – either my brain has chosen not to remember or my peril sensitive sunglasses went blacker than a priest’s socks.

After that I figured that a cycle ride around Yogyakarta was small beer so I put my name down immediately. The trip was actually organised by one of the travel agents on Jl. Prawirotaman – I’m just not sure which one it was because it was ultimately Intrepid Travel that it was done through.

Anyway, at 8am our steeds were ready to collect. There was some degree of choice – although I’m not quite sure how I ended up with the little purple number. Unsurprisingly it was exactly like riding a mid range 1990s mountain bike that’s slightly too small. It was the “too small” element that I found irritating, if it wasn’t for that I might have been tempted to try to sneak off with it…

Purple!
Purple!

My partner however selected one of a number of more traditional looking bikes. This was a decent piece of equipment – complete with V-Brakes!

DSC00784

Despite the variable age of the bikes they were all well looked after; there were no nasty noises, no frayed cables, the gears functioned without a hitch and the brakes were well adjusted and keen.

Now, you will have to excuse the lack of any real action photographs. You will understand that photography was rather low on my agenda for most of this trip – somewhat below not being run over by a truck, side-swiped by a motorbike or blundering into a drainage ditch full of monitor lizards.

Predictably, just about as soon as we set off there was a mountain of traffic on our tails waiting to get past. That was the strange thing though, it was waiting to get past. It wasn’t nibbling at our back wheels as if our presence on the road was an offence to the gods of motoring. It actually felt strangely safe – each vehicle waited until it was safe (well, safe for Indonesia) then gave a quick “pip” of the horn to warn that it was overtaking. It was all rather civilised and not at all the chaos I was expecting to have to deal with.

The fact is that in Indonesia the roads are full of all types of vehicles that travel at different speeds. In a modern car you’re perpetually behind something slower. So this is what road users in Indonesia expect – they don’t get frustrated with slower moving traffic because even if they can blast past this one they’ll only get a few yards before they come up behind something else slow moving.

In the West we seem to think that we have some sort of right to drive at – or slightly above – the speed limit. As a cyclist the fact that this is not the philosophy of Indonesian road users at all was really very refreshing indeed.

The other refreshing thing about the traffic is that the flows move and change, adapting organically to changing circumstances. Throw out a signal and you can feel the traffic start to adapt, sometimes to the extent that a space appears for you to move into. Road users in Indonesia cooperate with each-other because that way everyone gets to their destination faster.

I was genuinely surprised, I would definitely cycle in Indonesia again, in fact I’d rather cycle in Indonesia than in some European locations and certainly more than in the USA.

So in between marvelling at the road systems we dropped in on a few home industries – this model is big in Indonesia. Small, generally family run concerns just big enough to buy machinery and make some use of economies of scale.

DSC00707

First we discovered how tofu is made – in an anonymous looking farm building all of about 15m square. It was mighty hot in there though – not a job I think I’d have much stamina for. Nevertheless it transpires that making tofu is really quite simple. I may have to try it at some point!

DSC00708

It was on the way to the rice fields when I noticed just how easily I’d dropped back from my modern disk / v-brake “3 fingers round the bar, 1 finger round the brake lever” to the old “3 fingers round the brake lever, one round the bar” that was needed with cantilevers. The importance of this fact is almost entirely due to chickens. In Indonesia it appears they have taken the place of the pheasant as the creature most likely to hurl itself under you wheels for no good reason whatsoever.

DSC00715

Then our guide started complaining about his tan lines. I’m not quite sure who wins, I think we’re about even…

DSC00723

So I now know a lot more about the production of rice and it’s labour intensive stuff, lots of people working long hours in heat that never really drops below 30 Celsius.

DSC00731

Cohabiting with the rice fields are people making bricks. There doesn’t seem to be much set-up as it were, just a couple of buckets and a frame. The bricks are left to dry out in the sun before being “burnt” as our guide described it.

DSC00740

The final home stop of our trip was at Kwt Rahayu‘s modest home, for a lesson about tempeh. Soya beans are wrapped in banana leaves and left in the sun to ferment naturally. The result is partially fermented soya cake – it’s still got some crunch left to it and the fermentation process develops a wonderful flavour.

DSC00767

DSC00763

Kwt Rahayu has a lot of trophies and some of them at a national level – “almost all”, explains our host, “for tempeh”.

DSC00755

We then dropped in on a batik “factory”. Actually it was a showroom and this always makes me a little suspicious about the finances and the working conditions. They weren’t at all pushy however, a fact that gives a certain amount of confidence.

By now it was late morning and it was beginning to get hot so we headed back into the city. On the way though we spotted another of Indonesia’s home industries…

DSC00773OK, so they’re not actually building planes in small commercial shelters, this is actually for training. But it’s still rather a strange thing to come across on the outskirts of Yogyakarta!

By the end of it all I was really quite fond of my little purple mountain bike, I really didn’t want to have to give it back. I didn’t want to get on the plane back home either – Indonesia is a spell-binding country, I could have easily spent a couple of months there, not just a couple of weeks.

2008 Called: It Wants Its Time Back

Reading Time: 3 minutes
To Finish First, First You Have To Finish
To Finish First, First You Have To Finish

I’ve just been bitten. It’s one of the standard strap-lines of software development teaching – do it properly first time, don’t hack it and think you can fix it later. When the deadline is looming however sometimes we have to make a business decision;

  • Do we drop the feature (or some other feature)?
  • Do we miss the deadline?
  • Do we hack it in the full knowledge that we may be bitten later?

Fortunately for us software developers, technical debt is beginning to be understood by business managers. I explain it to non-techies like this;

I could hack it and get the feature in, but if I do that might be storing up problems for the future. Next time we want to add a feature in that area or even fix a bug we might have to unpick the hack and implement it properly before we even start on new work.

Hacking it now is borrowing time from the future – at some point we’ll have to pay it back and we’ll have to pay a wedge of interest too.

A few years ago I spotted a design problem; we’d assumed that something was always related to a location. In some circumstances however it could actually be related to a vehicle.

We should have changed the database to account for this, but that meant changing pretty much every component of an n-tier system and the deadline was busy whizzing past our ears.

There’s no way we could drop the feature so I had to find a solution. What I noticed was that I could cast a vehicle ID safely into a location ID without any possibility of coincidence. We could then detect in the DAL if it was actually a vehicle and generate a fake location record that actually contained all the information we needed.

A few years later and now we’re noticing that customers are putting a whole load more records in the location table than we’d initially thought and the indexing performance of the ID type is poor. So we want to change the type to something that indexes better. None of the options available allow us to cast a vehicle ID to a location ID any more.

Since those early days the problem has been exacerbated by the fact that more and more code has been piled in on the assumption that you can safely cast a vehicle ID to a location ID.

So we’ve now got a considerable amount of code to unpick, we’re going to have to do the redesign and reimplementation work that we (perhaps) should have done in the first place, then we can start looking at improving the index performance.

In situations like this it’s easy to look back and curse the past decision. Go into any software development office and you’ll almost certainly find a developer (or several) swearing at the screen, decrying previous decisions as “stupid”, “short-sighted”, etc.

I know I’ve made mistakes, I know there are situations where I’ve chosen to incur technical debt where there were better alternatives available. On this occasion though I made the right decision – we have time to repay this debt now. Back then we most definitely did not.

So “never hack it” isn’t a rule, it’s a guideline. But you have to be aware of the business consequences if you choose not to follow it.

Just Say No! – To Empty Exception Handlers

Reading Time: 3 minutes

Not a Fan of the Empty Catch
Not a Fan of the Empty Catch

We’ve been having a bit of trouble with a small piece of software on a customer site. It’s just not working and there are no indications of why. So I cracked open the source and found a light dusting of these…

try
{
    SqlConnection = new SqlConnection(connectionString);
    //the rest of the Sql code
}
catch (SqlException)
{
}

Oh look, it’s an empty catch. An empty catch is about as justifiable as a goto – I’m not saying that you should never use one, just that you need to have a good hard think about what you’re actually doing and consider carefully whether or not you’ve got the structure of your code correct.

The case against is easy to see above; although it may be normal for one type of SqlException to be thrown and ignored (perhaps as the result of a deadlock), you can’t guarantee that every SqlException is thrown for that reason. In the deadlock example your code should more follow this pattern.

try
{
    SqlConnection = new SqlConnection(connectionString);
    //the rest of the Sql code
}
catch (SqlException ex)
{
        if (1205 == ex.Number) //deadlock (see http://technet.microsoft.com/en-us/library/cc645603.aspx)
        {
            //we ignore deadlocks 
        }
        else
            throw;
}

This tests the SqlException to see if it’s a deadlock and ignores it – if it’s not a deadlock then it re-throws it.

I would argue however that this doesn’t go far enough. Exceptions should not be used for flow control which means that the very fact an exception has been thrown indicates that something has gone wrong. If you’re not going to handle that in code then the very least you should do is enable someone to find out that it happened.

There are many logging and debugging gizmos out there (such as log4net) and they usually contain a concept of levels of information. It would be a rare circumstance indeed where an empty catch was preferable to a minimal debug log, e.g.

try
{
    SqlConnection = new SqlConnection(connectionString);
    //the rest of the Sql code
}
catch (SqlException ex)
{
        if (1205 == ex.Number) //deadlock (see http://technet.microsoft.com/en-us/library/cc645603.aspx)
        {
            log4netInstance.Debug("SomeOperation caught Sql deadlock => {0}", ex.ToString());
        }
        else
            throw;
}

If this exception is normal and the app is expected to cope then you can turn the “Debug” level of logging off in configuration. If the app starts having problems then you can turn the “Debug” level back on and see all the exceptions that are being thrown. A subtle little thing like this can make a big difference.

On the one hand we have a quick phone call to one of the customer’s IT pros to explain that their recent security audit has resulted in them accidentally revoking the app’s authentication to the DB and that perhaps they’d like to reverse that.

On the other hand we have you, the developer, being sent to the customer’s site where you have to explain to some very senior people why your application has stopped working and what you intend to do about the poor quality, unreliable piece of software that you’ve supplied.

Never underestimate what can happen to your app on a customer site: Murphey’s Law is one of the fundamental underpinnings of computer science.

Sky Ride

Reading Time: 6 minutes
Grundisburgh Dog
Grundisburgh Dog

I’m a lone wolf cyclist really. Cycling is one of the things I do when I need to get away from people. Sure I’ve ridden with groups in the past but I’ve always found them kind of annoying – everything that should be simple suddenly becomes complicated. Even with off road groups. If I fancy a short detour past a nice café to pick up a bun and a cup of tea I want to just do it and not have to convince a load of other people that it’s a good idea.

I like it when it’s just me and the mountains. Unfortunately I live in Suffolk, which means it’s just me and the slight undulations.

Sky Ride

Sky however are – and have been for some time – supporting a major initiative to get people cycling. It’s called Sky Ride and basically it’s a lot of organised rides all over the country, led by small teams of volunteers. So as a supposed cycling advocate I thought it was about time I stopped being so antisocial and looked into this properly.

The first thing I would say is that if you’re looking at getting into cycling, or picking up a bike again after a long break then Sky Ride is absolutely ideal and you must go on one immediately.

There are three levels of difficulty. Sky Ride is supposed to be a recreational cycling initiative: I’ve had a look at some of the routes and these levels definitely refer to normal human beings rather than Lycra clad loonies.

  1. Easygoing: suitable for anyone including children, even a complete novice who’s only got basic cycling skills and may not be the fittest person on the planet.
  2. Steady: ideal for someone with a basic level of fitness and some cycling ability. May contain hills.
  3. Challenging: whilst these might not actually be that challenging for the aforementioned Lycra clad loonies they’re certainly not rides you want to tackle if you’re a little unsure about your fitness or ability.

All you have to do is plug your preferred location into the web site and it’ll suggest rides (and dates) near you. You can then go ahead and book your place on one.

Incidentally, if you are just starting cycling don’t be frightened by apparently large distances. I wouldn’t be thinking about any 50 mile excursions right away but 5 miles is perhaps a little over half an hour’s ride (at a similar effort level to walking for half an hour).

There just happened to be a 13 mile long “steady” ride near me at a time when I was free so I put my name down.

Joining the Ride

I left plenty of time, so I was about 15 minutes early when I turned up at the rendezvous point – the car park of a local sports centre. There were three people in a prominent position in blue “Sky” cycling tops clutching bikes who looked quizzically in my direction. “Sky Ride?” I asked. “Yup,” said Carol, checked my name off the list and handed me a free high-viz Sky bib. “You don’t have to wear it,” she said, “but it can be kind of useful if we get broken up.” We chatted a bit and I chatted to other riders as they arrived.

There were people with a range of bikes and wearing a range of kit. Some people had the full Lycra ensemble and bikes costing an arm and a leg. Some people had tracksuit bottoms, trainers and a basic high street mountain bike. Most people were somewhere in between. There was even a chap and his young daughter on a tandem, which I thought was exceptionally cool. Refreshingly there was no snobbishness, nobody was sneering at anyone else’s bike or judging anyone because they were wearing last year’s fashion.

Most people, but not everyone, chose to wear helmets. Under 18s have to in order to go on the ride but for adults it’s optional. That’s kind of an important thing to mention because if you don’t have a bash hat it doesn’t mean you can’t come.
Do invest in a pump, a spare inner tube and water bottle however. These are considered essential items. More info here.

The Protocol

Just before we set off the ride leaders explained the protocol. One of the ride leaders would always be at the front, one at the back and one in the middle. The group as a whole always rides at the pace of the slowest rider, except on climbs where we wait for the slower riders at the top. At junctions the ride leaders would organise it so that we’d all go through together.

There were times when the group would need to bunch together and times when we could – and perhaps should – ride two abreast. They’d tell us these things at the appropriate times but would appreciate it if we shouted the orders onwards as someone at one end of the group can’t always shout loud enough to be heard at the other.

Echoing information is pretty normal in any group, you can often hear ripples of “car back!” going from the back to the front or “slowing!” from the front to the back.

Armed with this (and a few other safety instructions) we hit the road.

Trouble at Mill

Important Arterial Infrastructure Road in Suffolk...
Important Arterial Infrastructure Road in Suffolk…

Things started off pretty well, once we’d got through the first couple of junctions and had the first couple of orders we’d all settled in. Then we went through Tuddenham St Martin. For a newcomer the climb out of Tuddenham is quite an up. One of the less experienced members of the group had a bit of a dizzy spell so one of the ride leaders stayed back with him while the other ride leaders found us found a safe place to stop. Meanwhile she established any possible medical problems and fed him energy gels and rehydration liquids.

He was right as rain after that: it was probably just a case of him going out too hard and too fast on the first hill. It might have been a bit of a shock too when all the more experienced riders streamed past him at the start of the climb and he may have felt some social pressure to try to push too hard.

It’s one of the simple facts of cycling though that sometimes, at that time in that group, you are the slowest rider. Even the fittest cyclists have off days and end up at the back. Waiting for people is part of the general protocol of cycling, so if you are on a group ride don’t feel the need to rush up hills, just make sure you get to the top. The more experienced cyclists will often be secretly glad of the break (especially the very, very experienced ones if you catch my drift).

As an aside it’s perhaps worth mentioning that one of the reasons groups break on hills is because of different “rhythms” of climbing technique. Quite simply the techniques people use to climb differ and disturbing someone’s technique by forcing them to go slower than they want to can mean that a hill they would have found easy suddenly becomes rather difficult.

The former patient showed no sign of a relapse, but the ride leaders and some of the more experienced riders checked in with him every now and again just to make sure.

From then on we set about bimbling around the beautiful Suffolk countryside at a comfortable pace, chatting and riding. Sadly I neglected to get it together to take any photos, so the two I’ve used here are actually from another ride this week on a similar route. One which it turned out I didn’t know quite as well as I thought and I accidentally took an 8 mile detour. You won’t have that happen on a Sky Ride!

Confidence

What struck me more than anything else was the level of organisation and professionalism. The ride was well organised, the ride leaders were friendly and chatty but assertive when they needed to be. They managed the pace and looked after the group effectively and efficiently. It all gave an air of confidence, that we were safe, we were being looked after. Even for an experienced cyclist this was encouraging and despite being a lone wolf I actually thoroughly enjoyed the experience.

So this is going to be the first Sky Ride of many I feel, although next time I think I may pick a “challenging” route. I like a nice bimble every now and again but “steady” was just a little bit too bimbly for me…

The Free Hardware Fairy: Windows Surface RT

Reading Time: 5 minutes
Surface RT: It's a Nice Idea.
Surface RT: It’s a Nice Idea.

I was on campus at Hull last week and pretty much as I walked in the door the ICT dept. jammed a Windows Surface RT into my hands. “It’s useless,” said a man in a Where Would You Think t-shirt, “enjoy!”.

I was quite excited about getting a Surface RT however, mostly because a lot of our customers do significant amounts of business when out-and-about. Developing for Android and iPad is a bit of a pig for Seed as it’s basically a Windows software house (although we do do it). So having something based on an ARM processor but that runs Windows must be a pretty major innovation for us, right? I mean most of our apps are pure .NET so you’d think it would be child’s play to get them onto the Surface RT right?

Oh no. It transpires that only Microsoft’s own apps are allowed to run on the desktop and yeah, most of our apps are for the desktop. In fact, without jailbreaking it, the only way you can get your apps onto the Surface RT is through the Windows Store.

Now I understand, from a technical point of view, why Microsoft may want to do this; Windows is still in a transition. Microsoft don’t really want anyone fiddling with that old Win32 stuff because it’s dangerous, but a lot of legacy apps still need it. Whilst Microsoft continue to provide access to the Win32 interface those applications aren’t going to stop using it. So they need to break the chain.

The reason it’s dangerous – indeed something that’s bugged Windows from the start – is that they’re trying to retrofit security. Basically Unix-like operating systems, of which iOS and Android are, start with the premise that as a user (or application) you’re not allowed to do anything. You are then granted permission to perform certain tasks. You can’t get at the soft underparts of the operating system unless you’re a superuser and in the case of iOS and stock Android devices they’re not letting you do it (although you can “root” most Android devices to get such access if you really want).

Windows however started life assuming that the user could do anything they wanted, so security simply wasn’t included at the start. This means that Windows has a heck of a lot more holes to plug than the other two platforms, and they’re not there yet. To clarify, we’re not just talking about hackers and viruses, but a lot of legitimate programmers have been using stuff in Windows – often because there’s no alternative – that can actually destabilise the system. Opening up the device to anything that would cross-compile for the ARM processor would mean they had exactly the same problems on the tablet as they do on the desktop.

Essentially we’re all still paying for a cock-up that Microsoft made in 1981.

There would be a half-way house though, as I mentioned before most of our apps are pure .NET which in theory can’t do anything nasty. The fact that RT is not open to pure .NET apps makes me scratch my head a little. I can’t help but wonder if perhaps the current .NET runtime (for desktop) isn’t quite a clean as it ought to be.

So, as a software developer I find the Surface RT frustrating, it has a desktop that I can’t use, I can’t write Windows Services (even in .NET), the only thing I can do is write store apps and even then it’s subtly different to writing a Windows Phone app.

Microsoft themselves have done a reasonably good job of getting their own apps onto Surface RT (e.g Skype, Lync) and there’s a smattering of the top apps that you’d expect to find – Facebook, Twitter, Kindle etc. but that’s about it. There is a woeful lack of applications for the platform. Not only that but even apps like Facebook and Twitter seem limited compared to their Android counterparts. They’re clunkier, it’s clearly that less attention has been paid to them.

So the big advantage of the Surface RT over an Android or iPad tablet is that it runs full Microsoft Office. That’s it. That’s the advantage and to be honest some of the alternative office packages that are available for Android are pretty good. In every other way it’s a worse platform than Android or iPad.
It even needs its own special charger rather than using Micro USB – although to be fair it uses a 12V supply whereas USB is 5V.

Annoying
Annoying

What Microsoft have done is to pretty much guarantee failure of the platform.

Still, it must be useful for something, right? I mean it’s actually quite a good piece of hardware. They detachable keyboard thing works really well too.

It’s certainly better (hardware) than the cheap Chinese Android table I got a couple of years ago. So why don’t I run Linux or Android on it? Well I can’t because Microsoft have locked the bootloader. Before you scream “no fair!” almost everyone does these days, Android included. Some Android manufacturers however release unlocks for their bootloaders. HTC for instance generally do when the device is a couple of years old (via htcdev.com). I can’t see Microsoft doing this.

In conclusion, the Surface RT is not the platform for development that I’d hoped it would be; it’s not very useful for us to develop apps for. Yes it has Office which makes it a useful device to have around for meetings etc. and it’s a lot easier to get in hand luggage than a laptop. It lacks the I/O though to make it a useful presentation device and there’s no chance of running up Visual Studio even if I could put up with it running terribly slowly. For me it cannot replace a laptop for anything other than the most basic business use.

There are cheaper Android devices that are far more capable – and the lack of Microsoft Office on them isn’t really much of an issue. There are x86 based tablets that run full Windows and, whilst they might be a bit slow, are actually viable as a travel alternative to a laptop. Microsoft themselves even produce the Surface Pro series.

To cap it all off the lack of apps puts it way behind the competition as a social and media platform so I’m really struggling to find a use for it. I’ll let you know if I find something a little more involving of technology than this…

Surface RT: Not Even a Good Doorstop
Surface RT: Not Even a Good Doorstop

The Corsa SRI

Reading Time: 3 minutes

We use hire (rental) cars at The University a lot which gives me the chance to drive a range of different cars. Today the hire car fairy brought me a Vauxhall (aka Opel) Corsa SRI. In yellow.

Yellow Peril!
Yellow Peril!

I was quite looking forward to driving it because it’s really popular with car modders, so presumably there’s something a bit special about it, right?

Err, yeah. I found that out really rather quickly, but first let me tell you what’s good about it.

It feels like good value for money. I’ve driven a few cars that feel like pieces of agricultural machinery with a few pieces of friendly plastic Blu-Tacked to them. The Corsa is pretty solid, well put together. Obviously there have been compromises but there’s nothing that rings out as glaringly cheap. Nothing agricultural.

The equipment is clearly a bit of a compromise. Boxes have been ticked, but the features are often difficult to use. Cruise control for instance, it’s there but it’s a bit of a battle compared to more upmarket Vauxhalls.

Nevertheless on the road the car is direct and feels very well connected to the road, you can throw it into a corner with confidence and know that you’re not going to be constantly fighting understeer.

It’s clearly relatively cheap and if you’re not expecting top class then it’s relatively well equipped.

But Where’s the Engine?

Thanks I suspect largely to the gearing the car is very nippy around town. I don’t live in a town, I live in the middle of nowhere. Sure it’s pretty nippy on narrow country lanes too but as soon as you get out onto wider roads you realise that only the first two inches of throttle pedal travel actually makes any difference.
I don’t think I’ve ever spent so much of a journey at 4500rpm trying to smash my foot through the bulkhead screaming “MORE POWER! MORE POWER!”
I can accept peaky small engines that have no guts outside the power band – I drive a Japanese car after all – but this engine has nothing in the power band either.

The Stereo is Woeful

Precisely No Features
Precisely No Features

A new car that doesn’t have DAB, USB or Bluetooth. Really? Just a CD player and AM / FM radio. Surely there are laws against this kind of cruelty. I wasn’t in the mood for Radio 4 and I’m not old enough to listen to Radio 2 so I whipped out my hand AUX cable and plugged the phone in. No matter how I tweaked the settings though I couldn’t get it to sound good. So when I stopped for a cuppa I tried tweaking the considerably more extensive graphic equaliser on my phone. Nope, the speakers are clearly made of cheese.

Clearly a Good Candidate for Modification

So to summarise, we have a car that handles well, is comfortable, is of generally good quality and comes with a good basic level of equipment. There’s a lot of potential for improvement however, almost everything could use a step up to the next level but I’ll single out the stereo (woeful) and the engine (was there one?) for particular attention. Coincidentally these happen to be the first two items that modders seem to want start tweaking…