Sunday, 23 December 2012

The Bundling of Ideas

There is a short cut we humans like to take when we need to classify patterns of thought which I call the bundling of ideas.

To highlight the concept I want to use an ancient example because it contains ideas most of us are not too familiar with. My example is from about 500AD in Byzantium when the Emperor Justinian ruled with his queen Theodora. In those days the selection of emperors was somewhat chaotic and Justinian came from left field. His wife was a courtesan or possibly a circus acrobat, not necessarily what you'd think of as queenly material. However these two made quite a pair. Justinian seriously expanded the empire which up until then had been in decline. Theodora caused some scandals partly because she favoured elaborate wigs and the clergy at the time believed good Christian women should cover their heads lest angels, looking down from above, be tempted to lust.

At the time there was a hot theological debate going on about the nature of Christ. No, this wasn't an ivory tower thing. There's an account by someone of the time who refers to going to buy some fish and getting into an argument with the fish monger. There were riots. Everyday people were passionate about this issue.

And the issue itself? It came down to whether Christ had two natures: human and divine or one nature: a combination of human and divine. Would that keep you awake at night? Me neither, and I'm Christian. The first view was called Duophytism and had been endorsed by the Council of Chalcedon in 451. The second view was called Monophytism and regarded as heresy, except in Egypt and parts of Syria where it was widely accepted. Most Christian churches today are at least nominally Duophyte but I'm not sure anyone cares that much.

The Empress Theodora seems to have been a Monophyte and her husband, Justinian, spent a lot of effort tryng to reconcile the two while holding firm on the position of the Council of Chalcedon. So people easily understood that Theodora was Monophysite and Justinian was Duophysite which was a little simpler than the truth.

Then there were the chariot races. These were the big sport of the time, everyone supported one of the four teams: Blue, Green, White and Red, though the last two were smaller and often teamed up with the others. So we can mostly consider the Greens and the Blues. Who you supported affected everything, including how you dressed and who you hung out with. And if you were Blue you were Monophysite, Green you were Duophysite, as well as a number of other political and military views. The people you knew well supported your team, your views, everything.

It probably wasn't quite that simple and there are always blurry edges to these things, but the factions were real. It meant that just by looking at someone, the way they were dressed, you could tell who their friends were, their position on a subtle theological question (which was really important) and whether they supported the Empress or the Emperor.

This is what I mean by the bundling of ideas.

We still do this today more often than we ought, and unbundling the ideas is like unpicking a stubborn knot. Take the notion of left and Right. By convention if you are left wing you support the following:
  • Gay marriage
  • Evolution
  • Worker's unions
  • State ownership
  • Lots of laws
If you are Right wing you support:
  • Traditional marriage
  • Creationism
  • Laissez-faire economics
  • Private enterprise
  • Fewer laws
Or do you? That's the problem with this bundling. People make assumptions. We have a minor party here in NZ which is right wing economically but has been left wing socially until fairly recently. Then, because it is a right wing party, it attracted other right wingers who were more socially right wing and shifted policies a bit to accommodate them. But those new people arguably only joined it because they assumed the bundling aligned with their own ideas. There is no real connection between social right wing ideas and economic ones that I can see.

In some parts of the world announcing you are Christian will label you as economically and socially right wing, anti abortion and pro gun ownership. In other parts of the world the same announcement will label you as a left wing radical, probably pro abortion and supporting gay marriage. You can see that, in these days of world wide communication over the internet, it is easy to say exactly the same thing to a lot of different people from different places and backgrounds and get very different results.

Sometimes people are more helpful. I saw an avatar yesterday that included a Bible, a gun and a US flag. There's a bundle that is probably fairly explicit, though I might have that very wrong. Do I really know this person's position on gay marriage? Abortion?

The answer is to be not too lazy about assumptions, try and be polite to everyone and try to make your own position clear without being too tedious about it. You might also ask yourself if you actually do hold the views you think you do, or did they just arrive in a bundle with something else?

Saturday, 15 December 2012

Interesting family connections.

I spent a pleasant afternoon hunting down some details about my long dead grandfather and the results are IMHO pretty interesting.

Anyone with any history has heard of the 'Great Depression' of the 1930s. Several countries associate it with a great leader who implemented Keynsean economic initiatives to rescue their hard pressed populations. In the USA it was Franklin D Roosevelt, in Britain it was Ramsey McDonald. In New Zealand it was Michael Joseph Savage. Working class homes here often had a photo of Savage on their wall and the man was looked on as an economic saviour. It helped that he was a quiet, modest sort of person and came to power around the time radio was widespread so his voice was heard in every home in the country on a regular basis.

But my research is not about Savage, it is about his Finance Minister Walter Nash. Nash obviously played a key role in any economic decisions and eventually became Prime Minister after Savage in 1957.

The family story I was following up on was that my grandfather, John Stych, was a friend of Nash. My mother told me this, and I think she said she recalled him coming to the house. She also said that her father had written to Nash when they were in dire financial straits during the depression and they all believed this letter was responsible for the introduction of a sickness benefit.

There were employment schemes at the time whereby the government made work for men (always men, actually) so that they could support their families. But my grandfather was too ill with tuberculosis to work. I gather the new benefit made all the difference.

I wondered if there was any truth at all to this story and I find there is, at least a little. There's no evidence that my grandfather wrote to Nash, but he certainly knew him, and quite well.

Walter Nash and his wife Lottie
Nash arrived in New Zealand in 1909 from Birmingham and he lived in Brooklyn, Wellington. He was a Sunday School teacher at St Matthew's Anglican Church and he was a member of the Church of England Men's Society (CEMS). He wore a CEMS medallion on his watch chain for much of his life.

He bought part shares in two tailoring shops, a business that eventually went bad and in 1913 he moved to Palmeston North and worked as a commercial traveller for a wool and cloth merchant.

So we have Nash in Brooklyn between 1909 and 1913 and involved in the Anglican Church. John Stych also lived in Brooklyn and was also a keen Anglican. Initially the family attended St Peter's because St Matthew's was not formed until 1909, after that they are recorded helping out there at church fairs etc. John's sister taught Sunday School at St Matthew's but John himself taught at St Peter's. That's him in the third row from the front, 4th from the right. He was, however, very close to the Rev R H Hobday, vicar of St Matthew's.  Hobday got him into missionary work in Africa and he gave his address when he returned on furlough in 1916 as c/o Rev Hobday, Brooklyn because his family were dispersed by then.

Since these two, Stych and Nash, were so involved in the same church they would have certainly met. Not only that, John's parents came from Birmingham, so there would be a connection on that level as well. John's father died in 1904 but his mother was still living and also attending St Matthew's.

John Styche's CEMS medallion
Then there is the CEMS connection. I have no family information suggesting John was a member of the CEMS but they typically wore a small Celtic cross on their watch chains, as did Nash. Photos of both John and his brother William show a similar cross on their watch chains. I even have John's cross in my possession. I only recently learned it was his. I should add that the CEMS was also something Hobday was very interested in.

There was a CEMS meeting in 1915  at St Matthew's with a lecture on "The Foundation and Principles of Socialism" by Mr H E Holland, presided over by Hobday. Holland was a key figure in the Labour movement, and the first leader when the Labour Party formed the following year. John was in Africa at this time and Nash would have been in Palmeston North by then. But this meeting throws light on the intellectual climate of the CEMS meetings at St Matthew's under Hobday.

When he returned from Africa John worked as a 'traveller', which normally means commercial traveller, as did Nash and both were involved in the textile industry at some points. John was a draper's assistant and we already noted Nash owned a tailor shop. The CEMS seems to have worked as an old boy network to some extent with members helping each other find employment, so it is quite possible John used that network to find work, and also possible Nash was part of that.

Finally in February 1921 Nash was convicted of importing seditious literature. John's brother Thomas was similarly convicted in early April. The circumstances are different: Nash was working as a publisher's agent in Wellington and said that the literature reached him as samples which he did not intent to circulate. Thomas claimed the literature had been thrown to him from a boat (he was found with the material on Hobson Street wharf in Auckland). Neither story sounded likely to the respective judges and the men were convicted and fined.

These two cases may be unconnected but the fact that they occurred so close together in time and that Thomas Stych and Nash may well have known each other too is interesting. The police at the time would, of course, have seen no connection. Thomas Stych was also convicted of refusing to serve in the army during WWI, something many of the founding core of the Labour Party shared.

This got me thinking about my grandfather's time in WWI. He returned from Africa in 1916 and he offered himself for service and was pronounced unfit. We don't have any information as to just why he was unfit. In Africa he had suffered from blackwater fever but he had several months to recover by then. The officer who signed this off was William Longhurst, his future brother-in-law. John subsequently went back to Africa to resume his missionary work. In practice he was seconded into helping administer the steamers on Lake Nyasa (now Lake Malawi) which were by then under the control of the British Military. I don't know what his thoughts were about serving in the army, but he seems to have managed to avoid it one way or another.

Tuesday, 4 December 2012

Madura Perspectives Revisited

You can now run the Madura Perspectives Manager demo online from OpenShift. The details of just what Madura Perspectives Manager is can be found here. Once you've logged in (admin, admin) it looks like this:

The rest of the demo script is in the previous post and I won't repeat it all here.

The last few days we have been making some minor tweaks to the Madura Perspectives Manager. MPM is an awkward beast to explain but easy enough to demo... provided I can get you to build it and set it up yourself. So maybe not so easy.

The problem is the bundles, and I solved that yesterday. I'm going to explain how here. Madura Bundle are jar files that load into an application dynamically, ie while the application is still running. If you know about OSGi you'll be nodding about now and maybe wondering why I don't just use OSGi. Good question. But not for this post.

Madura Perspectives Manager loads entire sub applications from bundles as described in my earlier post and the bundle manager normally works by scanning a directory for bundles it can load. It does this every few seconds (configurable, of course) and while this can work fine in a real environment it doesn't play too well with CloudFoundry which we use to host our demos. Even if you install it yourself you'll need to edit the directory location and copy the jar file bundles into it. A bit clumsy really.

What OpenShift would be happiest with is storing the jar files inside MongoDB or similar and they happily support that kind of thing. This would work fine but it seems a lot of messing about for a demo, so we bypassed that and kept it simple. The bundles are stored in the relevant Github projects so we modified the bundle manager to pull bundles from a list of URLs. It doesn't show the dynamic-ness of the bundle management, but it does show everything else.

This has got me rethinking some things about that dynamic-ness. The rule is that if you add a new bundle then users who are already logged in will not see it. So if you are a user working in sub application A and there is an update to A then you will not see it, and the application you are using will continue as normal. This is what we want, we don't want users disrupted by new versions in mid flight.

Once you log out and back in you'll find the new version of sub application A. Naturally the people who worked on the update of that sub application have made sure any work you saved under the old version is compatible with the new one.

It is actually possible to do two other things as well.

You can delete a sub application. I'm wondering if this is ever a good idea, though. Deleting a sub application is supported by the bundle manager but in practice it screw up anyone who is using it, so the plan is to modify the bundle manager to simply disable deleted applications  rather than actually unload them. That means that newly logged in users will see the option, but it will be disabled.

You can actively decide to pick an older version. Let's say the bundled sub application you are using generates a quote for some complex product you want to sell a customer. The customer worked through a quote with you yesterday and you gave him a price good for a month. This morning, when you log in and look at that quote, you find the price has changed. This is because marketing have just released their new pricing plan or rules or numbers or whatever. It is all bundled into the application and the latest application gives you a different price.

But that is okay. If you are able to select from all the versions you can get the one you want. In fact if you fetch that quote from a database you can have the system automatically pick the right version for you based on the date on the quote. We haven't added this feature into the Madura Perspectives Manager demo, it is complicated enough, but it is certainly a natural extension and easy enough to implement.

Thursday, 29 November 2012

Directed Questioning

This announces the latest version of Madura Rules, part of my Madura toolkit.

Madura Rules is a rules engine that plays nicely with Java objects, the rules refer to fields in the objects and enforce validation and derive new data. This all happens transparently, and with almost no api calls. You just write your code to talk to your objects and the rules just happen.

But this was missing something I used to have in an earlier engine I built about 20 years ago: Directed Questioning.

Let's say you have a really complicated data input scenario, something like an insurance form or a tax form. These things often have a lot of questions on them that users don't have to answer because the questions don't apply. There are a lot of statements like 'go to step 17 if you answered NO' and so on. If this is paper based (and it usually is) then the amount of paper is somewhat intimidating, even when the user might only have to answer 5 questions or so. And finding which five questions out of, say, 200 is awkward.

You could build a software system that codes this kind of form, of course, and people have. But they are hard to maintain. Adding a new field or two means you have to review what circumstances a user has to supply the information and then adjust your navigation and flows to ensure the right users answer those questions.

Directed Questioning bypasses that by using the fact that the rules engine knows all the relationships between these questions and can dynamically customise the UI as the user enters the data. Here is a simple example that calculates your BMI.
formula: Customer "BMI"
    bmi = weight / (height * height);
formula: Customer "Height Metric"
    height = heightMetric;
formula: Customer "Height Imperial"
    height = (heightFeet * 0.3048D) + (heightInches * 0.0254D);
formula: Customer "Weight metric"
    weight = weightKilos;
formula: Customer "Weight pounds"
    weight = weightPounds * 0.453D;
These are the rules the engine uses. They happen to be all formula rules. We can use if/then rules as well as constraint rules if we need to, but I am keeping this example simple. The first rule is the standard BMI formula and the rest of them allow the user to enter in metric or imperial units, depending on what they prefer.

All we are really interested in is the value of bmi. We can tell the rules engine to
go figure it with a simple Java call. What actually happens then is the engine looks at the rules and sees that it needs a value for height and weight. To get weight it needs to fire either the third or fourth rule, it doesn't matter which at this point.

So it tries the third rule, and that means it needs a value for weightKilos. There aren't any rules that deliver weightKilos, if there were the engine would go look at them, so it asks for weightKilos. That's the first question the user sees.

This user is happy to answer that and inputs a value, so the engine derives a value for weight. It does not have to ask for weightPounds and it knows that. Now it is looking for height.

To get height it tries the second rule and asks for heightMetric. This user is happy with metric weights but not length and he doesn't know his height in metres. We can tell the engine we 'don't know' that value and it will look for another way to get the height.

There is the third rule which needs heightFeet and heightInches so it asks for those, figures the height and then it can work out the BMI.
So, of the 5 possible fields to enter this user was only asked for 4. If he had known his height in metres it would have been only 2.

The questions are asked one at a time, because each answer might eliminate the need to ask the next one. The API is designed to be architecture friendly, it actually queries the engine for the next field to ask for, and how it is asked (ie what technology is used to ask it) is up to the application. My current preference for these things is to use Vaadin and I have a working example on rhcloud you can look at from your browser. Log in as admin/admin and click the BMI button on the main screen.

Tuesday, 27 November 2012

John Chilembwe

John Chilembwe is regarded as something of a martyr in Malawi today. I found a reference to him in a book about Malawi history I've been reading.

My grandfather was a missionary in Malawi 1913-1919. He wasn't a priest or a preacher, he mostly built houses, schools and churches for the mission effort. He wrote some notes of his time there and I've put them here. So I have been reading a few books about the early missions in Malawi, wondering if any of them refer to my grandfather (only one so far, he was a minor player in the scheme of things). But I did find references to John Chilembwe and my grandfather's comments on him were interesting to compare.

First, we have to know about Joseph Booth, an Englishman who arrived in Malawi in 1892 with his nine year old daughter. He writes of his arrival:
"...I reached this spot, weary, sickly and weak. My little daughter was with me... A drenching rain was falling. Several Angoni men were sheltering under a grass roof they had constructed on the boughs of that tree. They had made a fire there and were crouching round to keep warm... Yet these men out of their good nature got up instantly and insisted that myself and little child should sit around the fire while they stood close to the trunk of the tree in the rain to get what shelter they could."
Booth, unlike some of the English he met in Malawi, seems to have understood that the Africans were just as human as he was. As such he was regarded as a radical by some members of the English administration. His catch cry was "Africa for the Africans" which did not go down too well with the English settlers. I'm not sure just what the other missionaries made of him, but they were likely enough more sympathetic. A driving concern for the English missionaries was the slave trade. Raiders did good business in Malawi raiding villages and selling their captives on the coast. The missions' general approach was to generate enough local industry to make the slave trade comparatively unattractive. The Administration's approach leaned more towards using force to stamp out the trade.

Life for the missionaries was pretty tough. They seemed to die like flies over there, subject to tropical diseases unheard of back home. My grandfather came home alive, but with illnesses that blighted the rest of his life.

Back to Chilembwe, though. Booth hired Chilembwe as a 'house boy', basically a domestic servant, and proceeded to educate him along with the others in the mission he set up. He seems to have been a star pupil and part of the family. Booth's daughter, Emily, wrote of him years later with sisterly affection. Booth and he travelled to the USA in 1897.

By 1915 Chilembwe was running his own mission, helping with the rather too many disputes the local Africans had with the English settlers, and campaigning against the idea that Africans should be helping fight the English war (WWI). And that is when the real trouble started. The actual incident seems to have been quite minor and, though it was represented as an African uprising, it resulted in about three English deaths, several injuries and Chilembwe and his friends were pursued by police and shot. There was also an unsuccessful and possibly unconnected attempt to raid the munitions depot in Blantyre, one of the key towns in Malawi. The whole affair seems to have been over in a week apart from a Commission of Inquiry launched by the Administration which concluded Chilembwe had been treated unfairly and the issues he had raised were valid.

But where communication is tricky rumours can spread. My grandfather was quite some distance from the centre of it but he heard of the Chilembwe 'movement'. Here is what he says:
It was while I was building the church at Kayoyo that the "Chilembwe" movement started probably influenced by some enemy in Britain. Anyhow it never got beyond Blantyre where it started. The order was sent out for every boy to kill his master and thus free the country of whites so that Chilembwe would take charge. It was some while before I heard of this through the mail and I asked Stephen about it, had he heard about Chilembwe?
"Why didn't you tell me, were there not enough of us here to protect you?"
"Why should I cause you worry?"
Can we trust a native? And this was the man who was allotted the job of cutting my throat according to Chilembwe.
Of course it is more likely that Chilembwe issued no such orders, but it also looks like if he had the missionaries at least would have been safe enough. The Africans seemed to have a pretty clear idea of the missionaries' motives, understanding that these were good people doing their best, and they responded accordingly.

Ref: Malawi: History of the Nation, Bidglai Pachai

Friday, 23 November 2012

A Dangerous Idea

At the age of about 12 I was exposed to a dangerous idea. It would be a better story if this went all dark and blighted my life from then on but it didn't and I turned out fairly normal. But the idea still lurks and surfaces fairly often. Since it didn't blight my life I guess it won't blight yours either so I will share it.

There is a series of books by James Blish called 'Cities in Flight' about whole cities converted to space ships and lifted off the earth along with their industrial factories etc. They move from planet to planet selling their services to the inhabitants, all originally from Earth as well. Blish took this story into the far future. At times it is a metaphor for the Okie phenomenon in the 1930s and the cities are the itinerant tramps hawking their labour for meagre wages. They even call themselves Okies sometimes. In the final book he makes several sly references to Gulliver's Travels and around then the dangerous idea emerges not as a key part of the story, just as an aside. The image shown here refers to the second book in the series.

I should add that these books contain fascinating ideas, especially for something written in the 1960s, but the characters are mostly cardboard things with no depth, so not great literature.

In the last book one of these characters expresses a belief in something called 'stochasticism' which is a term that crops up in the real world in discussions about quantum effects and the unpredictability of various other things. Blish's character explained that his stochasticism view held that everything is random and the laws of the universe are an illusion. In a big enough universe that is truly random chances are there will be pockets of something that looks like order. We just happen to live in one of those pockets.

For now.

The orderliness of our particular part of the universe extends to as far as we can see, which is a long way but the universe is really, really big so that doesn't help. It is ordered at this point in time, for this few million or billion years or so, and that might change at any moment and everything we know would fall apart.

Is this just nonsense? Don't we know better?

Well, as far as I can tell, we don't. Science has done very well for us and scientific method is probably the most useful thing we have invented, but even without being very scientific we make assumptions about the consistency of the universe all the time. People who thought the earth was flat still assumed the sun would rise in the morning. Consistency, see? And we observe this consistency over and over, so it must be right, mustn't it?

Well, only if we can justify something called induction, which is what we are doing when we assume the observations we made in this lab on this date will apply in a different lab on another day. David Hume seems to have been the first to point out that we have no logical basis for that assumption. No matter how many times we observe something, as Karl Popper pointed out, there is always the possibility that the next observation will break our theory. And if the universe is really random we have even less basis for it regardless how often we have observed it before. It is an awful thought and if you tried to base your life on it you probably wouldn't bother getting out of bed in the morning. This is the dangerous idea.

To make things a little more awkward the rules of science hold that theories must be kept as simple as possible. This is Ockham's Razor. Which is simpler: the completely random universe which has natural pockets of seeming order, or the one with universal laws? It seems to me that the former has less to explain itself.

Of course no one does actually believe this. We continue making our observations, noting patterns in the way the universe seems to work and telling ourselves these are universal laws that apply across all time and space. Naturally some places have more gravity or more radiation etc than others so we're not saying all time and space is exactly like Earth is today, but we are saying that we can figure this out because there is something to figure out.

No, don't tell me there must be universal laws because they've been observed empirically. If you still think that you haven't been paying attention. Go back and read what I said three paragraphs ago. 

If I have explained this well enough you should now be convinced that, while the dangerous idea is a valid enough and, by the rules of science and philosophy, can be argued as preferable, there is no way you are going along with it. You are damn well going to believe in those universal laws because the alternative is simply awful.

Fine. I completely agree. I'm not going to endorse it either. I'm going to live in a universe where there are universal laws even if I can't prove there really are universal laws. Which reminds me of something:

I’m on Aslan’s side even if there isn’t any Aslan to lead it. I’m going to live as like a Narnian as I can even if there isn’t any Narnia.” - C.S. Lewis, The Silver Chair”

Writing Portable Mobile Apps

Since the rise of smart phones writing phone apps is all the rage, to the extent that it almost seems we have forgotten what we used to do before there were smart phones. There are tons of desktops and laptops out there armed with browsers, but my local bus company wants me to download an app to my phone so that I can basically look at their web page. So do my local weather service. Even my phone company wants to do this.

All of the above present the same information on a web page which I can access just fine using the browser on my smart phone. So why do they go to the trouble of developing mobile applications to do the same thing? Notice I said 'applications', plural. They typically develop for iOS and again for Android. The two use different APIs and different development tools so this does mean double the effort. But it doesn't stop there.  There are Blackberry, webOS, Symbian and Windows phones out there too and, assuming you think they are all worth supporting, you'll need to rewrite your application for those as well.

And all to present the information on your web page. Really?

The reason they want to create an app is probably one or more of the following:
  • The newly hired batch of geeks are getting bored
  • We need to present a cool image to our customers
  • We can monetize the app (ie sell it) through the various app stores.
Different app writers will have different reasons. The weather service I mentioned above wants me to pay for their app, so they're clearly into the 3rd reason. The phone company and the bus company give their apps away so I'm thinking one of the first two reasons applies to them. Or maybe once I use their apps I will find add-on pay for features, or maybe they have advertising.

None of these reasons are bad reasons. But they do seem a little pointless when what you end up with is just the same as the web site.

And there is a better way anyway.

In the early days of desktop computing the entire program, including its UI, always ran on the desktop machine in much the same way smart phone apps do today. Many desktop applications still work like this (MS Office, for example).

But these always need you to install something locally and as the Web took off applications started to be entirely web based. This was always difficult to manage well partly because the browsers were fairly dumb and partly because there were so many different technologies to integrate (html, javascript, JSP or ASP, backend languages like Java or C, databases etc). Then you had security and failover to manage.

But we moved on to smarter techniques using AJAX that could build really slick applications that were web based but look like desktop applications. Google's GMail and most of their other services are examples of this approach. They also published GWT, the framework they use to build their applications. There are others in this area too such as Dojo and Echo, so Google doesn't own this space.

But the interesting thing about GWT is it means you can forget about those other technologies and just write Java. The Javascript is generated transparently (including the code to handle cross browser inconsistencies). But it doesn't stop there.

Vaadin developed their product on top of GWT and Vaadin's approach simplifies the GWT stuff, provides a ton of out-of-the-box widgets (which you can add to) as well as a rich framework for building heavy duty Java applications at the server side. Using Vaadin you don't even have to worry about GWT, though you can tweak CSS stuff if you want (and you probably do want to) to add a richer look and feel. But CSS is just look and feel, not function. The CSS team can go work on their own while the application programmers work on the real stuff (oops, I just insulted all the CSS experts out there, sorry).  This separation is important. It ensures the two groups do not trip over each other.

Vaadin applications are designed for desktop browsers. How does this fit into your mobile ambitions?

The answer is Touchkit, a Vaadin add on which has widgets that look and behave like smart phone controls. The default style is iOS but you can tweak that for other environments, and you're still using the same application.

What about monetization? Oh, yes, you still want to see your app in the app store and charge for it. Okay, you can do that by surrounding it with a webview, but it is still the same application and it is still running in a browser. There is a small gotcha there in that Apple are a bit wary of webview applications, although they are allowing some. Also I'm not sure about serving up advertising in an application in a web view, probably fine, but it might need some work to get it going.

That gets you a browser based app that is installed like a native mobile app, but you only have one copy to develop and maintain. What are the downsides?

There are a couple. First, you do have to have a server running the application somewhere. So if you were developing a stand alone Tetris game this solution is not for you. Having said that Vaadin's Touchkit supports going offline (actually I think it is riding on GWT's support for this) and synching subsequent changes afterwards, so you're not quite tied to the internet. But you do need a server.

The second one is that you are still running in a browser. What the browser cannot do your app cannot do. So, location services are okay because HTML5 browsers support that. Camera access not so much. There is a work being done in this area with a product called PhoneGap which seems to understand how to drive all the mobile platforms I mentioned early on in this post. Actually they list 7. It takes HTML, Javascript and CSS and turns them into an app for the target platforms, including accessing the accellerometer, compass, camera etc. They do appear to have had some issues with Apple about whether their apps can be in the app store. But these seem to be resolved.

Vaadin applications, as far as the client is concerned, are precisely that (HTML, Javascript and CSS and the client doesn't know about the Java back end code running on the server). More about it working with Vaadin here. There is ongoing work in this area, but we should be able to write totally portable mobile web based applications.

Thursday, 8 November 2012

Courier Blues

I need to rant about couriers. Does anyone else get the same run around I do?
Okay, maybe my situation is a little odd.
We have a house in the country and an apartment in town. We only want to get our mail in one place so while all the neighbours in our country road have RD boxes (Rural Delivery, like a letter box, the postal service delivers most days) we get our mail in town.
Most of our neighbours in town get their mail in the boxes in the communal area and the postman puts it there most days. Well, when we first moved there we had mail stolen from those so we got a PO Box. This means there's a kind of mini locker at the nearest post office and the letters are placed in that. It has a lock, so it is secure.
Long ago I worked for the postal service here in NZ and I know a bit about the process. The PO Boxes are open at the back to allow sorting. But the public aren't allowed into that area (there are locks on the main doors etc) making it quite secure.
So, our usual way of getting mail is from a PO Box and this works perfectly every time, never a problem.
But often when I order something online they won't take a PO Box number because they want to use a courier. That's when the problem starts.
I have used the apartment address but they inevitably try and deliver while I'm out (and I am out a lot). It is pretty much impossible to arrange a time like 'anytime Monday morning', they'll come on Friday, leave a card and come back on Monday afternoon.
I've had cases where I stayed in all day and called the courier when no one arrived. They told me he had arrived but no one was there. Did he go to the wrong door and I didn't notice? It's a tiny apartment, that's not actually possible. They didn't say but I gathered the driver had gone to the wrong suburb that time.
After a few of those I took to using the work address. There is always someone on the desk in office hours so should be fine right?
Most of the time yes. Then a birthday present I ordered for Mrs failed to turn up. I still don't know what happened but I called the vendor and they promptly sent a replacement which (not their fault) arrived too late.
I tried putting a 'hold for pickup' instruction on it. That worked fairly well, but I did have to drive out to the airport for it and they had a bit of trouble finding it. I gather no one ever does this and they didn't really know how to handle it. After much humming and hawing a nerdy guy came out from the bowels of the place and said 'are you Roger Parkinson?' yes 'okay, I know where your stuff is'. And he did.
Okay, so this time when confronted with 'no PO Box' I decided to try the country address. Let's see what happens.
I can track FedEx on the web so I know it arrived in Auckland within two days of them sending it (good). Then it stayed in Auckland 'in transit' for a week. I phoned their Auckland office to find out what they were doing.
Well, they'd passed it to a local courier who were 'waiting for instructions'. It seems they decided the address I had given did not exist. The lady from FedEx added '...but I can see it on Google... okay, I will call them now.' I should make clear I think FedEx did a good job here, their only mistake was passing it to the local courier.
The local courier phoned me and I gave them directions to my house and I told them not to deliver on Thursday morning because I'd be out. He wrote this down (or I think he did, he asked me to repeat several points slowly etc).
Next morning (Thursday) I got a call from the local Rural Delivery lady asking where my house is. I could tell her, but I also pointed out that I was out. And I had told the courier company this only yesterday. They had passed the box to the RD people with none of my instructions. A quick conversation with the RD lady established where I live (she knows the road well, she drives it most days) and that I would be home the next day and, sure enough, she brought it this morning as promised.
The weak link in this chain, obviously, is that local courier. It seems that every interaction I ever have with any local courier (there are lots of them) is a problem. FedEx were fine and the RD people were fine.
I have removed several vendors from my list of people to buy stuff from because they won't ship to my PO Box, and this doesn't happen too often now. So
  • if you are a vendor think hard about who you ship through. Your stuff might be great, but if your customer gets this run around he will buy elsewhere, even if it costs more.
  • if you are an international courier (looking at you FedEx, but there are others) think hard about who you pass your stuff to. The locals can let you down and make you look bad even when it isn't your direct fault.

Sunday, 23 September 2012

Fake Rat

Calling it a Fake Rat sounds better than calling it a bird scarer. We have a problem with swallows, actually they're 'welcome swallows' but they're not very welcome. I quite like them and they have a cute way of finding nooks and crannies around our house to make nest. That would be fine, except that they leave trails of crap down the walls under the nests. So they're not welcome.

We play this kind of game. They start a nest. I knock it down. They start another. I knock that down. All the time I'm wishing they could go nest somewhere else. All the time I'm sure the female is saying "but... I've got these eggs going!"

Then I thought of this. It is a fake rat, well it looks a bit like a rat. The fur is rabbit fur actually, because I'm not going to skin a rat and I had a spare rabbit skin from one I shot a while back. It is wrapped around a cardboard box and inside the box are some electronics which, every half hour, light up the blue LEDs and make a growling noise.
I'll put it near their current favourite spot tonight. They'll get settled in there for the night and this thing will 'come alive'. Then they'll fly off, disturbed. But they'll come back, of course. They'll take a look at the rat and decide it is harmless and settle in again... and then the rat will come alive again.

This will go on All Night.

I would like to insert an evil, maniacal laugh just here.

Now, I know it doesn't look that much like a rat, but they're just little birds and I reckon it will be scary enough for them. A furry thing with glowing eyes that makes a noise has got to be alive, right? And they go to a lot of trouble to build their nests where furry things cannot reach them. Clearly wherever this thing is a bad place for a nest. Job done.

The electronics is pretty simple. I have an ATTiny85 IC still on a breadboard in there with two LEDs (in parallel) and a speaker that plays three notes. It is very like the Laser gun I built earlier, except I've taken much less care with the build and there's no way to recharge the batteries. I have three AA batteries in there and they are easily accessible when I need to replace them.

The main difference is that, to save battery life, I use the sleep mode for the ATTiny85. There are several sleep modes available and the one I have used puts it into state where it uses very little current while it is waiting for the half hour to come up. Most of the time, of course, this is what it is doing. The sleep mode uses the watchdog timer which seems to be just an internal oscillator that, after so many cycles, delivers and interrupt to the main program. It is fairly limited, the max sleep time is about 8 seconds. So it wakes up more than the birds know. For these silent wakeups I just increment a counter and go back to sleep.

How well this all actually works is will be tested tonight.

Wednesday, 22 August 2012


Not the snake, the language.

Python is a popular scripting/interpretive language that uses object oriented structures, so as a scripting language it is pretty sophisticated. Not only that it is easy to run it from Eclipse, including debug mode which allows you to step through the code, examine the variables etc, much the same way I can do with Java and C++.

I'm having to learn this because I'm building a small add-on to Gramps. Gramps is the genealogy program I am using to maintain my family history data. I have to recommend Gramps. It is open source, and it can spit out a fully interactive website with the information when you've got everything entered. These two points mean I know I can easily pass the data around to people who just want to view it (the html) or edit it (the open source thing). I'm not sure I'll actually do the latter (well, maybe when I am dead) but the former is good if I just want to dump it all to a CD and hand it around. The recipients don't need to install anything, just run their browser. I even threw a copy onto my phone and it works fine (so I have it with me if I need to look up something).

Gramps also has a nice plugin architecture so it is fairly simple to write some extra code and add it to the system... as long as you do it in Python because that is what the system is written in. I am doing something to the database (more on that in a moment) so I assumed I could just write a bit of Java, talk to the database with JDBC and do what I need. I don't actually have any interactive stuff, just a batch database update.

Except Python has built in a non-SQL database which doesn't really handle JDBC, and Gramps, naturally, uses this because it is right there. It is also, I suspect, more suitable for this kind of data than SQL, though that's always arguable either way. Regardless, it is not happy with JDBC. There are Java interfaces to it, but they look a bit messy and with the entire source of Gramps loaded up I have plenty of Python example code, so I set to work.

What I did was not hard. It is a plugin to scan all the media objects (photographs, in practice) and pull the Exif data from them. Depending on what is present I push the relevant Exif data into the Gramps database. The idea is I keep titles, descriptions and dates on the actual image files. This means they show up in various photo management tools automatically and they don't get detached from the photo. But I also want them to show up in Gramps.

Gramps doesn't do this for itself, or not the way I want anyway. This is partly because the Exif data structures are very loose and they can't cater for the endless combinations to figure out what is where. My images are tagged explicitly by me so I know what Exif tags are being used, so I can write the code to map what I want.

It was not a big deal. But do I like Python?

Not really. It was easy to set up in Eclipse and fairly simple to pick up the syntax, and the debug tools work well. But I prefer a compiler phase that finds my syntax errors (especially when I am making a lot as I feel my way through a new language). Python finds these at runtime, which means I have to keep working through my test sequence. I could have shortened this with a unit test, and I would have done that with Java anyway, but that was more figuring out and I got lazy.

So I'm not keen to switch to Python any time soon, but I can see why people like using it. I will stick to Java and C code for now.

Thursday, 9 August 2012

Steampunk laser pistol

This is the second generation pistol. The one I made last year was both simpler and more complex. Last year I cut a handle out of MDF and used a Teensy 2.0 to generate a noise when the laser was active. It looked okay. This one is better.
  • 3D printed handle, nicer shape, more detailing etc
  • Blinky LEDs to make it more exciting
  • A glass barrel. It isn't actually glass, but it looks like it.
  • A charger circuit so I don't have to take it apart to charge the battery
  • A more interesting noise
  • Using an Attiny85 rather than a Teensy
The handle was a big deal but it was worth it. I used blender to model my shape in 3D and then got shapeways to print it for me. Fairly pricey when I add freight all the way to NZ, but so much better than MDF.

Here's the handle as it arrived.
It might not be too obvious that this is a gun handle but it will come clear soon enough.

The handle is hollow to accommodate the battery and it has a ridge around the edge so the pearl inset I'm adding will fit snugly. There are two curly bits I'm rather pleased with. It took some fairly fine detail work in blender to get those right and they turned out pretty good. Getting detail that fine with MDF is, of course, impossible. MDF just tears and you cannot carve it.

The next step was to paint it so it looks like metal, and you can see how those curly bits look pretty cool. This picture also shows the pearl insert in the handle.

I also want to mention the raised lozenge shape just above the lower curly bit. There's a wide hole through the structure underneath that and I wanted to add some strength to it at that point. I think the material, now that I've actually handled it, didn't need it. But these are the kinds of decisions you can make with 3D.

The next picture shows the insides of it held in place by a stand with a couple of claws so I can work on it. There is a round green perf board and you can see the speaker dangling underneath and to the right. Opposite that are some stiff wires with the LEDs attached. There's another LED, a green one, on the board. This will light when the battery is charging. You can just see the black corner of the Attiny85 IC between the two claws. I chose this angle to conceal my bad soldering. I have a really good soldering iron, it isn't that. I solder badly anyway.

The code the Attiny85 is running is in a google code project. That also has the 3D model, which I've made public on shapeways too.

The almost finished gun looks like this. Now you can see the barrel in place and you can also see the round perf board sitting at the base of the barrel. The tiny speaker slots into the top of the handle just behind the barrel. It almost fit but I had to scrape back some material with a knife. The actual laser is that brass thing in the tip of the barrel. All it really needs is that and a battery to be a laser, which would not be nearly as much fun. The barrel is made out of a clear plastic bottle, so it won't break if dropped. The trigger is a red push button.
The copper coil is decorative only and the LEDs are kind of tangled in amongst it.

Here's what it looks like in action:  

So far the gun has the electronics exposed. What it needs still is some leather to cover the end of the barrel to give it a more steam punk look. The glue around the actual laser at the barrel tip is a bit messy too so I made some covers for both. There's a thin strip of leather at the tip now, and a copper cover at the other end. So now it looks more steam punk.

The original of this, just so you know where I got the idea, is this classy Derringer. which is a good deal more ornate but much the same shape.

But I was also influenced by one of the guns on Warehouse 13, especially for the glass barrel.

Neither of those are real lasers though.

Sunday, 29 July 2012

Eclipse AVR, Attiny, Teensy

This is how to get your Eclipse AVR to play nice with Attiny and with Teensy.

The Arduino IDE is a fine piece of work, but I've been using Eclipse for years and I prefer that for a number of reasons. If I had to pick just one I'd point to integrated source code management, but there are other reasons too.

So, with this in mind I decided to take a look at the Eclipse AVR plugin. Initially it didn't work too well for me until I could Jan Baeyens' fork of the plugin which has some extra setup built in. As far as I know the instructions there work out-of-the-box for proper Arduino boards, but since I don't actually use them I had some extra work to do. Specifically I wanted to get Attiny ICs and Teensy boards working in this environment.

Let's summarize my environment first.
  • Ubuntu 11.10. It probably makes little difference if you are running Windows or Mac, but just so you know.
  • Eclipse Indigo (Indigo Service Release 2 Build id: 20120216-1857)
  • Arduino Eclipse Extensions 1.2.3
  • Attiny85
  • Teensy 2.0
  • Teensy Loader 1.06 (from
Building for the Attiny

To flash the Attiny I have the setup described in an earlier post, so my laptop has a USB cable plugged into a Teensy which acts as an ISP to flash the Attiny85. This works fine from the Arduino IDE so from a hardware point of view we are all good.

If you installed Eclipse and Jan Baeyens' plugin already you probably pointed to your existing Arduino files. You need to change that. The best approach is to clone your entire Arduino IDE directory tree. That way any changes won't mess up your working Arduino IDE (and, believe me, these changes will mess it up otherwise). Just how to point the plugin to another directory is described further down.

I'm also assuming you already set up your Arduino IDE to handle Attiny (again, see my earlier post) and the Attiny files are in your cloned Arduino IDE. We will refer to the cloned Arduino IDE directory tree as and we will ignore the original one from now on.

The next steps need you to look in this directory structure
There are two things to do here. Under /hardware/arduino/cores I have added two links, one to the attiny files (/hardware/tiny/cores/tiny) and one to the teensy files (/hardware/teensy/cores/teensy). If you are using Windows, which doesn't support links, you can just copy the directories over. They are not very big.

The second thing is to edit the boards.txt file and add the Attiny entries. You can just copy the entries over from /hardware/tiny/boards.txt but there is a catch. Each variant has a line like: @ 1 MHz  (internal oscillator; BOD disabled)
this is always the first line and it describes the board variant. The problem here is the semicolon. So when you copy/paste the entries into your boards.txt file make sure you remove the semicolon like this: @ 1 MHz  (internal oscillator BOD disabled)

Also add the following entry

This ensures it uses the Teensy as an ISP. If you're using something else you might need a different setting.

Once you have your boards.txt edited open up eclipse (restart it if you already had it open) and check the Windows->Properties.
Make sure your Arduino IDE path is your cloned path.
Make sure your avrdude configuration file is properly set up.

Finally make sure the paths look like this (your arduino file can be in a different place, of course)

After that you can just follow Jan Baeyens' instructions for creating projects etc. If you create a new project for your Attiny it will automatically create a library project holding the core Attiny files. When you build your project it will reference the library project (well done, Jan!). You want to build your library project first, though or things might get confused.

To flash the code to the Attiny once it is built the plugin calls avrdude, which is what the Arduino IDE uses as well. Open the properties for your Attiny project (mine is called Blink).

On the lower right you can see the AVRDude option is picked.

Check the avrdude command line looks roughly like this.

And that should work. A build should build the project, create the hex file and launch avrdude to send it to the Attiny through your ISP which, in my case, is a Teensy.

Building for the Teensy

If you already worked through the Attiny setup then you are okay to proceed, if not then do the following steps described more fully above:
  1. Clone your Arduino IDE directory
  2. Install the plugin and point to the cloned directory.
  3. Make sure the paths in your Windows->Preferences are as described in the Attiny section.
Now edit your /hardware/arduino/boards.txt file and add the Teensy board descriptions. You can find the descriptions to paste in /hardware/teensy/boards.txt, but these have a lot of stuff you don't need. You can safely remove all the *.menu.* entries. Also add the cpu speed eg:
(this is for Teensy 2.0, but you can run Teensys at different speeds if you want)

Take a look in your teensy directory (). In mine there was a file called main.cxx which was empty and another called main.cpp which had a main() method defined. The compiler picked up the first but not the second and then the linker complained of a missing main() method. Just remove the main.cxx if you have one.

Restart Eclipse and create a project for your Teensy. This can take a while, a couple of minutes or so, while it creates the library.

Build the library.

Open your project properties and, as with the Attiny, ensure avrdude is enabled in the toolchain but we are not actually going to use avrdude, we are going to use the teensyloader so we need to adjust the avrdude settings by changing project properties like this:
My teensy tools will be in a different directory to yours but you can adjust that. Notice that we are invoking teensy_reboot and also notice that under Expert Settings we have removed the ${FLAG} parameter.  Don't worry about the 'All options' box, that will not be used.

There is one more trick before this all works. The teensy loader, that's the executable called just 'teensy' and is in the same dir as teensy_reboot, needs to be already running, so you need to manually start that. Once it is running teensy_reboot will tell the teensy loader to flash the teensy board with your new code.

Friday, 8 June 2012

Home grown business cards

For some years now we've been able to set up tiny companies, give them a fancy web page and make them look just like a huge corporation. As long as your customers are happy they don't care how big you are.

We see a similar thing with eBooks. Indie self-publishers have books in the big on-line stores like Amazon, iTunes, B&N and their readers don't seem to care that the books didn't come out of one of the big 6 publishing houses.

Part of this is that we can simply do so much more for ourselves. We can take photos (and videos) and edit them without a complicated darkroom, and we can even design and 'print' 3D objects if we want.

So why would we get our business cards printed by someone else?

I'm not knocking the people that do it, and I used to do this when I was in the corporate world. I never seemed to use more than a few of the minimum 200 cards they would print for me. These days I sometimes need a card with my contact details on it to give to, say, the person I'm dropping off something I want him to fix for me. He wants to know how to get back to me and it is tedious to do this verbally. So I hand him a card.

The information doesn't change very often but I know I only need a few of them and I wanted to play around with the design. Maybe I want a different design next time. Organising this on a page in OpenOffice (or MS Word, if you must) is trivial and printing them on a laser printer is just a button press.

But the story is a little more interesting.

I decided to add a Q Code to the back of the card, just because I could. Perhaps some of the print shops that do cards would do this, but I'm sure it would be tricky and expensive etc. For me it was quite simple, even if I expected it to be no actual use.

While I'm not corporate these days I did find myself in a business meeting earlier this week and when cards were being passed around I handed out mine. I had just a little concern that it wouldn't look quite professional because it doesn't have 'ltd' or 'inc' anywhere, just my name, phone and email.

"Oh, you've got a Q-code on the back!"

"We should have this on ours."

"Damn right."

And everyone was pulling out their smart phones and scanning the code. The way this works is the scanner software grabs the Q-code information and allows them to push it straight into their contacts. They instantly have my email, phone etc without having to type it in.

So my home grown business card was, in fact, quite a hit.

Tuesday, 29 May 2012

Spin Doctors

I have been collecting these for a while now. It seems the same patterns come up over and over. The specifics change, of course. And it isn't just politicians, you can find anyone using these from middle management to military dictators. There's probably a spin-doctor handbook somewhere that has the complete set with detailed instructions.

The report is out of date

This is when some report or finding appears condemning something they are responsible for. The response is to effectively claim they already fixed the problem, which might be true, or might not.

We're still in discussions

When a proposal comes to light that people are unhappy about. Often the people they are in discussions with are some large business who stands to gain heaps, but sometimes they are just talking to other politicians. This implication here is that we should not take this too seriously because it all might change and they are just playing with ideas. This is never the case, of course. If we stop yelling about it they will do the thing we fear. They might anyway. After this phase you often hear:

It's a done deal, we can't change it now

They don't say it like this, of course. They say it is a great deal that they have achieved by being such good leaders etc, but the deal is done. The documents have been signed and backing out of the deal would now cost an enormous amount of money to recompense other parties for broken promises.

There's been a cost blowout

Yes, even though they said the deal was done and we couldn't change it, we find that other parties involved can change what it costs. That means they can make us pay more (for this deal we never wanted in the first place). There is a variation on this one where they have to pay heaps (of our money) to sack some highly placed official because they are breaking a contract. The official is being dismissed for incompetence or worse, but we still have to pay a golden handshake for some reason.

I'm sorry you took offence

This is when they screwed up and can't quite wriggle out of something that looks like an apology. It isn't actually an apology, though. It just seems like one. To be fair I have heard at least one politician apologise properly saying something like "I screwed up. No, I can't blame anyone else, it was me and I'm sorry." But this is astonishingly rare. And that one I recall was preceded by an attempt to squirm out of it.

I will sue you for defamation

Sometimes they mean it, sometimes they don't. They seem to say it more than mean it though. And we have to wonder, when they don't follow through, if the defaming statement were true. There are other reasons not to sue (too busy, too expensive etc) but threatening it in the first place and not following through is the mistake.

I am absolutely and completely in support of motherhood and apple pie

Of course they are, we all are. But the question was: why are you cutting support for solo parents trying to feed their children? The trick here is switching or simplifying the argument so that the real point is obscured. We are supposed to nod in agreement because they've picked something we all support, so we agree with them, don't we? That's all fine then. There are endless variations of this. A recent local example was a politician who was responding to the suggestion someone had organised a discounted hotel stay for him (which could be construed as a bribe). He vehemently responded that he had paid for the hotel himself and it was on his credit card. Of course it was, we knew it would be. But that was not the accusation. It was, indeed, on his credit card and the rate was indeed discounted.
We can't afford it
No, we can't afford to do this as well as the other stuff we'd rather you didn't spend our money on. We probably can afford to do this if we stop doing something else.

Sunday, 13 May 2012

Playing a melody on ATTiny85

This is fairly trivial but I wanted to document the steps carefully.
The code is here and I flashed the ATTiny85 using the steps I referred to here.

The flash pins are still connected though only VCC abd GND are necessary to run the program because they supply power. I have 5V running across them which is coming from the Teensy which, in turn, gets it from the USB.

So save confusion I am going to refer to the ATTiny pin numbers with a #, so #4 is GND and Arduino Pin 3 is #2 etc.

So the speaker is connected across #3 and #4 (Arduino Pin 4 and GND). I tried several speakers ranging from 9 ohm to 47 ohm. Not a lot of volume but quite audible. There were no resistors or capacitors etc.

Once running the program plays a 7 note melody over and over until you get annoyed with it and turn it off.

But there is a critical step I missed the first time around. There are several implementations of the ATTiny core for Arduino. Not all of them support the tone() function because it is tougher to do on an ATTiny (so fair enough).  Anyway the one I used is here. That also lists a bunch of other implementations so there is a lot of choice. The readme tells you how to add it to your Arduino directory.

Once it is installed you do need to pick the right ATTiny85 option in Tools->Board. There are several and the one I found that sounded right was "ATTiny85 @ 1 MHz (Internal oscillator, BOD disabled)" The others are oscillating at different speeds which means the clock doesn't run at the speed you want and the melody is scrambled.

I have the melody working on #3 just fine although it is not documented in the diagram as a PWM pin. In fact #2, #3, #5-7 all work. This may be because the Arduino core I am using is not actually using PWM for the tone(). Not sure. It works anyway. The comments in the core code match the diagram above , ie #5 and #6 are PWM and no others.

Saturday, 12 May 2012

Flashing an ATTiny85 from a Teensy.

Here's the project: flash an ATTiny chip from a laptop using a Teensy board.

I've been using Teensy boards for several things lately and they are very good. But for one project it is overkill. Teensy boards can blink lights just fine, but so can ATTinys, and ATTinys are smaller and cheaper.

But flashing a Teensy is very simple, you just plug a USB cable into it and use the usual Arduino IDE to load your code directly from a desktop or laptop. Not so with an ATTiny because it isn't as clever. I could mention boot loaders and similar but I'd get out of my depth fairly soon.

To flash an ATTiny you need to use the SPI protocol and you won't get your laptop talking that through a USB cable, so you need something to act as a go-between, taking the serial data being sent down the USB cable, turning it into SPI and pushing it through to the ATTiny. Obviously this is complicated. The go-between is called an ISP (In-System Programmer).

People use a custom box for this such as a USBTinyISP or they use an Arduino board as an ISP like this. But my projects tend to be very compact which is why I tend to go for Teeny boards, the smaller of which is about the size of the end of my thumb. So I have various Teensys lying around but no Arduinos. Fortunately someone already figured out that you can use a Teensy as an ISP just like an Arduino (there are a lot of similarities between Teensy and Arduino, including using the same IDE).

So we're all good, except the technique didn't actually work.

Here is what I did to get it working:
  1. Follow the steps in this link (same as the one above), except I used a Teensy++ 2.0 and an ATTiny85 chip. These probably don't make much difference, but the pins have different numbers.
  2. I'm using ArduinoIDE 1.0 like he is, and the Teensyduino software. He's using the 64 bit version and I am using 32 bit version, again should be no difference.
  3. I used a modified copy of the ArduinoISP software which I've copied to here. I got this from here and changed the pins to work with the Teensy++ 2.0. So this version has the pins for the Teensy++ 2.0 (this is probably the big difference, ie what made it actually work)
  4. The program for the ATTiny is here. This is just a modification of the blinky example for Arduino. Again, just the pins are different from the Arduino standard.
  5. I did edit the arduino-1.0/hardware/arduino/programmers.txt to arduinoisp.speed to 9600. I forget where that was recommended. My modified ArduinoISP code has a similar change.
This is what my breadboard looks like:

 That's the Teensy on the left, with the USB on the extreme left. In the centre is the ATTiny85 and over on the right is a blinking LED. As you can see there are no resistors or capacitors, just simple connections (the two exceptions are the resistors needed on the LED). Some sites will tell you that you need other bits but I didn't for this.

The wires here aren't that easy to see but I've got GND on the top rail and VCC on the second rail. Near the VCC in the upper left of the Teensy we have MISO, MISI and SCLK across to the corresponding pins in the ATTiny. SS is connected to RESET on the ATTiny.

There is a green LED on pin 3 of the ATTiny85 because my test program will flash the LED attached to that pin. The blue LED flashes when the Teensy is flashing the ATTiny. I ought to add a red LED to indicate an error on pin 8 of the Teensy but I didn't. In fact both of these 'LEDs' are actually just one RGB LED.

I made the heartbeat flash the on-board LED on the Teensy so that I can tell the ISP code on the Teensy is running okay.
It is worth noting the options I used to upload the ISP code from the ArduinoIDE
The board selected is Teensy++ 2.0 and the Programmer is AVRISP MKII, which is just normal for Teensy programming.

But for the ATTiny we need this:
In this case the board is the ATTiny. There are lots of ATTiny options but any ATTiny85 does for this program. The Programmer is Arduino as ISP (and the Arduino IDE does not care that it is a Teensy instead).

This allows you to do everything from the Arduino IDE, as long as you set the options correctly. But ArduinoIDE actually uses avrdude to transfer the files to the Teensy ISP. This is the command it ends up using:

/home/roger/kits/arduino-1.0/hardware/tools/avrdude -C/home/roger/kits/arduino-1.0/hardware/tools/avrdude.conf -v -v -v -v -pattiny85 -cstk500v1 -P/dev/ttyUSB000 -b9600 -Uflash:w:/tmp/build4483866087774593940.tmp/BlinkAttiny85.cpp.hex:i

Your arduino directory will be different to mine but that is easy to tidy up. The reason this is interesting is if you want to use the gcc compiler to build your code instead of ArduinoIDE. You can then invoke avrdude manually using a similar command to this and the hex file will transfer down.

For a handy reference to the pins of the ATTiny and others use this.

Monday, 20 February 2012

Madura Bundles and data versioning

When building applications using Madura Rules we have some choices when defining the data for the decision tables. This is the data that tells the rules what combinations of settings work together. In the pizza demo this tells us, for example, what toppings and bases go with what pizza sizes. The data is not usually very large and is only ever read, not written.

In the pizza demo this data is held in an XML file stored as a resource on the classpath.

 There are two interesting issues to solve here.
  • Data stored in XML is all very well, but it is not so easy to maintain. This is especially true if you have multiple operators working on it at once.
  • There is no ability to version the data. Everyone using the application sees the same combinations and options, always the latest one. What if they want to see how things were last week? For example they might have saved a shopping cart from last week and want to order this week and the data changed meanwhile.
 As long as we keep things in XML we can solve the second problem quite easily with Madura Bundle. The XML file in the pizza demo is just a resource and distributing resources is supported by Madura Bundles. The idea here is that the application selects one of the available bundles (and the XML file is in the bundle). Extending the shopping cart example, when last week's shopping cart is picked it would hold the id of the bundle used to create it, and the application would pick that bundle before restoring the shopping cart. Everything would then work as normal.

You can add new bundles any time and new shopping carts would always pick the most recent bundle. Sometimes there would be a reason for an old shopping cart to run under the latest bundle, perhaps if we decided the old bundle was no longer valid for the business. Maybe it holds prices we don't want to support now.

The bundle can also hold rules and code if you want. You could put as much of the application you want into the bundle. Just where the balance should be is up to you.

Now, about that XML. Could we use a database? Actually yes. HSQL has a memory-only database option, including one that we can load from data held on the classpath. Naturally it is read-only and naturally you don't want to hold gigabytes of data in it. But you can use ordinary JDBC to access it, and that means you can use more interesting things like JPA on top of that.

To use this with Madura Rules you'd need to write some Java, namely you'd need to extend the AbstractDecisionTable to support the actual database structure you actually have. This is the simple bit. You define the data source like this:

<bean id="dataSource"
   <property name="driverClassName" value="org.hsqldb.jdbcDriver" />   <property name="url" value="jdbc:hsqldb:res:/testdb" />
   <property name="username" value="sa" />
   <property name="password" value="" />

This assumes you have a file called testdb.script in the top of your classpath. You can optionally add HSQL creates these files if you create a database using a URL jdbc:hsqldb:file:testdb (testdb can have a directory in front of it, this example uses the cwd).

H2 has a similar option, but I haven't used that yet so I'm not clear on the details.

Can we put the database into a Madura Bundle? There wouldn't be much point to this post if we couldn't, I guess. So here's how.

All you need to do is define the data source inside the bundle. I actually defined the data source, JPA entity manager and a DAO class as beans in the bundle. Note that the classes for these are on the classpath of the application. You don't have to bundle those classes (this is one of the differences between Madura Bundles and OSGi). The DAO class is exposed to my main application.

Because HSQL remembers the database name and assumes a request to open a database with the same name is the same database, you do need to ensure each bundle calls the database by a different name. This is simple enough

<bean id="dataSource"
    <property name="driverClassName" value="org.hsqldb.jdbcDriver" />    <property name="url" value="jdbc:hsqldb:res:${}-testdb"/>    <property name="username" value="sa" />
    <property name="password" value="" />

Using the symbol like this means the file this data source looks for is (in my case) bundle-1.0-testdb, bundle-2.0-testdb and so on. I also have to rename my testdb.script file to bundle-1.0-testdb.script etc.

Now this all works just fine in a simple test case except that HSQL adds a '/' to the front of the file name and Madura Bundles does not like this. I've added a small patch to Madura Bundles to overcome this, but it isn't quite enough to justify a release just yet. It will be in version 3.5.