Change and large systems

Warning: Undefined array key "HTTP_REFERER" in /home/sites/ on line 159

Generally, as systems get larger, it is more work to make large changes to them.

Building apps with an agile platform such as agileBase is very quick. However over the years, as new members of staff join, they may find it hard to work out how the system has been put together, which parts see a lot of use and which less.

Over the past months we’ve been working on addressing these things and we’re in the midst of designing some exciting further developments in that area too, but today I want to talk about the more technical aspects of making changes to very large and complex systems – what happens under the hood.

To understand why this is important let’s look at an example. Many businesses create calculations of component cost, VAT or other financial values, which then feed into many places of the system. Some customers have many hundreds of views, all referencing the same base calculation view.

What happens if we want to make a major change to that calculation view, such as adding or removing a field (gross and net figures as well as VAT perhaps)? 

What you may not know is that when you do this, the system automatically checks and rebuilds every dependent view. While this is happening, users and even the API are blocked from reading or writing data. Usually everything wraps up in a second or two and no one notices the time lag but sometimes, when there are hundreds of dependent views, it can take longer, causing a queue of waiting requests to build up and eventually time out, resulting in people seeing error messages.

Incremental view rebuilding

As of today, if the system works out that it’s going to be too slow to update everything in one shot, it will switch to updating dependent views one by one, in the background, allowing people to go on working.

Now, while a view is waiting to be updated, people will see a message like this:

The system will generally rattle through rebuilding all views necessary, so even when there are many hundreds, people should have to wait no more than a minute or so. And parts of the system which aren’t affected can carry on being used as normal.

Most importantly, these background updates will take a lot less resources while running.

We know that some administrators have previously found it necessary to wait until evening time to change views which many others are dependent on, to avoid the danger of overloading the system as lots of users tried to use a system at the same time as it was being modified.

Hopefully with this update, hard working system administrators can regain some of their work-life balance!

What else has changed?

A number of other improvements have taken place recently – here are some notable ones:

Comment fields usability improvements

The user interface for adding and viewing comments has been much improved, in particular comment boxes are now more differentiated from normal editable text boxes, making it obvious which is which.

Secondly, we’ve introduced ‘comment tags’. When a comment is imported from an email someone sends the system (e.g. to attach an email to a contact sales enquiry automatically by forwarding it or BCC’ing the system), it will get tagged with the sender/recipient of the email. Chaser responses will also get tagged and in future, we may use tags for other things, or let people add their own.

Dates in spreadsheet exports

When a view is exported to a spreadsheet, any date fields will now become proper dates in Excel, rather than text representations of dates, so you can quickly sort and filter them without having to do extra work to transform the cell type.

Record merging improvements

The record merging feature has seen a lot of use and has had some usability improvements and bug fixes. In particular, when a comment is produced due to one unique value overwriting another, that comment won’t be included in the comment stream on the homepage. These comments were polluting the comment stream when heavily used.

Inline editing improvements

Relation fields can now be edited when a view is in inline editing mode. Some customers are finding this indispensable for speeding up particular data entry tasks.