Wednesday, 21 May 2014

Deploying an Eclipse Update Site

I have been working on an Eclipse Plugin to support people who use my rules
engine and workflow engine. These both require you to specify stuff in a text file, rules in one case and process definitions in another. There is syntax to get right and the plugin provides a helpful editor that tells you about syntax errors, suggests things you might need to enter, colours keywords etc. It isn't nearly as smart as the Java editor in Eclipse but it should help.

Some people might be surprised I've done it this way rather than a pointy-clicky user interface that non-programmers can use to define their workflows and rules. But I have been around this stuff a while now and

I have yet to see a non-programmer actually write any rules or any workflow. Instead I have seen pointy-clicky user interfaces that are too dumb to actually do the job because some things are just too hard to make that easy.

Also, even if non-programmers did get directly involved, they would have to learn how to handle issues like version control and rollout which would mean they would turn into... programmers.

So these products accept reality and make it as easy as possible for programmers, especially Java programmers, to work with them. Hence the need for an Eclipse plugin that helps with the editing. I use Eclipse all the time, have done for years, partly because of the way people extend it with plugins. There are alternatives to Eclipse but one plugin is enough for now.

Actually writing the plugin is pretty simple. It took less than a week to throw it together. The writing and testing part just needs one Eclipse plugin project and you write the code and tell it to run. That launches a second copy of Eclipse with the plugin installed and you try it out, step through with the debugger etc. Easy stuff.

Deploying it however, that was much more complicated.

The way the people who designed the plugin architecture would have me do it is to create several different projects which are interrelated. I need one for the plugin, of course, another for the 'feature' which contains the plugin, another for the 'update site' which builds the repository other people can use to download from. If I use Tycho, which is what I'm supposed to do if I want to build with Maven, then I need another couple of projects as well. That seems too complicated, at least for a single plugin project. If I had several plugins I wanted to roll into one site then maybe, there would never be more than 3 'overhead projects' but...not this time.

Instead I cooked up an Ant script which builds everything in one project. That should make it very simple for anyone who wants to pull my source code from Github and try building it. One complication is that the build.xml file (which is what Ant looks for by default) kept getting deleted by the Eclipse wizards so I changed its name to build1.xml. The file is a bit messy still but it does run okay and it delivers an update site in the 'site' subdirectory.

That means I can commit the site directory and use the Github project as its own public update site. So people who don't care about the source code and just want to install the plugin binaries into their copy of Eclipse can point to there and get it. It took me ages to work that out. I was expecting to deploy it into Sonatype, which is where I deploy all my other binaries, but they are Maven binaries and Eclipse update sites have a different structure. Fair enough, but there were hints around that it was possible which made me keep looking too long. Anyway, this approach is easy and seems to work fine.

Thursday, 1 May 2014

Madura Workflow

Here's what I have been working on recently: Madura Workflow.

At this point your eyes glaze over, of course, but let me assure you that this stuff is really neat, even though you almost certainly don't need it.

Over the decades I've come across quite a few Workflow products, sometimes they're called Business Process Engines and such like. There may be a problem with specific definitions I've never got to the bottom of, anyway I was never happy with them. So here's my effort.

You use workflow when you're building the kind of computer application that needs multiple tasks done by different users, some of them external users, at different times, possibly over weeks or months. Maybe you need to handle a complex insurance application which needs several actuaries to look at different aspects of it, you might want to send parts of it out to external consultants and get a response back, some of those might be automated. The application's route through your system might need decisions made which influence its path. For example if some amount is larger than a cutoff point the application may have to go to a supervisor who is authorised to handle it rather than the lowly minion it would go to by default. It might need to send a credit check request off to the credit reference agency who have an automated response system, and they charge for each request so you want to avoid sending to them if you don't need to. Choices again.

The process definition works through the steps, but under that there are tightly integrated rules provided by my Madura Rules Engine. This means that you don't have to write code to make it all work. The rules fire automatically (tightly integrated, see?) and stuff just happens. This ensures the process definition is not polluted by a load of non-relevant logic. In other products like this they often are polluted, you see, and it means business rules are spread all over the place in the system, making it hard to find what you have to change. There are validation rules, process rules, UI handling rules and so on. In this they are all in Madura Rules.

We can send Web Service messages out to external services and process the responses. Again this is done with no code. We use Spring Integration to handle this at the moment, though the connection point to that is very thin and it could be replaced by something similar. Anyway, it means that, unlike many workflow systems, the process definition is not polluted by details of the messaging.

When you have a task that needs user interaction you can present a form generated from the data structure (again no code required) for the user to complete. Naturally the form is connected to an object that is monitored by Madura Rules. So you get all the validation and error handling etc you could want. You can, if you want something more specific, write some Java+Vaadin code for a specific form.

You'll gather by now that implementing a process is pretty simple because it is detached from the generally nasty business of defining application logic, UI code and messaging. That's the idea but there's another trick here too. In practice these kinds of coordinated processes can take weeks or months to pass through a business system. They might even get passed back and forth between two or more people several times. What happens during that time if you want to change the process definition, or the rules, or the form or the messages?

Obviously I wouldn't raise the issue unless I had good news. Madura Bundle allows us to package all of that stuff into a separate bundle file that is hot-swappable without having to restart the server(s). Existing process instances remember what bundle they started with and they keep using that. Newly launched processes use the latest bundle. Easy. So, for example, if that cut-off point mentioned above needed to change you could just regenerate the bundle with a new value.

If you've read this far you might want to take a look at All this stuff is open source and, except for commercial implementations of the rules engine, entirely free.

There's a UI which is fairly complete based on Vaadin, but the underlying workflow library is agnostic to the UI technology, so if you prefer something else go for it. I already said the messaging platform, Spring Integration, can be switched for something else and actually even Madura Rules could be replaced because it is implemented as a plugin to Madura Objects and you could replace it with another plugin.