Export controls

Near the end of last year, we introduced export notifications for sensitive commercial or personal data. Along with various other features, that lets administrators of a system be notified by email if a user exports more than a certain amount of data.

Well, this week’s update expands on that. As well as being notified, administrators can, per table, control how much a user can export at any one time, i.e. they can set a hard limit. This limit takes effect per export, so bear in mind it wouldn’t stop someone exporting an entire database of contacts for example, by doing lots of different filters and exports one at a time (e.g. all As, all Bs, all Cs etc.) but it would increase the effort and the administrator would get lots of notifications.

Remember, you can also set privileges to prevent any exporting at all for certain groups of people and/or views.


To set this up, go to the Build section in the agileBase admin interface. Then go to the manage tab for the table you’d like to control.

Make sure that one or both of ‘stores sensitive’ or ‘stores personal’ data are ticked and you’ll see two options. The first lets you set the limit at which admins will be notified of exports (e.g. 10 records). The second, which is the new option, lets you set the ‘hard’ limit. If someone tries to export more than this, the rows exported will automatically be cut off after this number.

The export limit applies to all views based on the table, in fact all views which any data from the table appears in. If a view uses data from many different tables, the minimum export limit will be applied when exporting from that view.

Please take a moment to think whether this will be useful for your organisation and set it up if so – we want all customer data to be as well protected as possible.

If you’ve any queries, don’t hesitate to get in touch as usual.

Big data

Last week we started talking about scalability in terms of the API and that got me thinking about what we’d do if and when we need to process orders of magnitude more data than we do currently.

Currently, the largest single table of data any one customer has comprises just over thirty million records. That may be a lot of business data in some contexts but in database terms it’s hardly ‘big data’. What about if we needed to store and work with billions of records?

A natural direction to look in would be to www.citusdata.com. As you may know, agileBase uses the open source PostgreSQL database. Citus transforms PostgreSQL into a distributed database. And it’s open source too. Their pitch is ‘Never worry about scaling again’.

Citus has recently been bought by Microsoft and is now available as Hyperscale (Citus), a built-in deployment option for Azure Database for PostgreSQL. It can also be used on AWS or your own hosting. So there are many options to allow scaling to many nodes to handle billions or hundreds of billions of rows, while maintaining high performance.

That’s for the future, but if anyone has an inkling of any projects they’d like to put forward which deal in high volumes of data a.k.a. ‘big data’ in the IT world, then drop us a line. We’d be keen to be involved.

Updates roundup

It’s been a while since the last product update on the 1st April, but that doesn’t mean we haven’t been hard at work developing the agileBase platform further. It just means we’ve been too busy working with new customers to do much blogging (plus taking the odd summer holiday to be fair).

However, there are some great new features recently released to tell you about, so let’s summarise them quickly:

WYSIWYG editing

This one had been on the roadmap for a while and some specific use cases from new customers pushed it onto the active pile. In short, when editing text in a large text box in the user interface, people can now easily format their content, adding bold, underline and italics for example.

To format a word, just double click it, or use keyboard shortcuts like Ctrl+B / Cmd+B to make text bold, as in any common wordprocessor. Here’s what it looks like in use:

Image extractor

This is one for the ‘experimental’ list – we’re releasing it as a beta feature and hopefully it’s really useful for some people, but bear in mind that beta label.

If you have an image field with type ‘identifier image’ in a table, when you type a web address into a text field into the same table, agileBase will visit that website to extract and store what it thinks is the main image on it – most often a banner image near the top, or possibly a logo. That can add a bit of interest if you have company information in a CRM for example. Of course, the system may not always be able to get the best image, or any image so you can always manually upload.

If you’d like to try this out, please let us know.

API scalability

We keep a keen eye on the performance and scalability of our services, starting with efficiency at the lowest level of algorithms and programming – after all if a single thread or server is efficient, that also makes adding more hardware (scaling ‘horizontally’ or ‘vertically’) much more effective.

However, most of this work to date has focussed at the level of the database or our own code. Many newer customers are heavily using agileBase not just to build business apps in their own right, but more of a data engine to power a whole host of inter-related infrastructure, for example transactional websites, mobile apps and feeding into enterprise-wide reporting.

All of this relies on the API – the mechanism by which agileBase communicates with other software. The long and the short of it is we’ve made some major improvements to increase the volume of data that can flow in and out without impacting other operations, such as making changes to the system at the same time, like adding a table or field for example.

We sat down with an expert (thanks Rich) to work out how we could increase scalability in this area. Some ideas have been actioned and some are in the bag to pull out in future as and when needed.

We hope these updates are useful to current and indeed future customers. Please feel free to contact us to get involved with the continuing process of improving the agileBase platform!

Visual refresh

For the start of April, we have a Spring refresh of the record editing interface for agileBase, making forms more friendly and breathing new life into the interface.

Continuing the previous work on form simplification such as ‘contracting blocks‘, this makes working in apps built on the agileBase platform an even more pleasant experience. There are various improvements to style and alignments, the most obvious being that form field outlines are changed from boxes to underlines, to reduce visual clutter.

Tab colouring

In addition, administrators have a new UI tool in their armoury – coloured tabs. Tabs on screen can now be colour coded, to draw users attention or give them information. For example, if a complex process required half a dozen tabs, you may want to colour them green as each step is completed.

Setting this up works on views, in a similar way to many other features in agileBase. From the parent table of the tab, create a view which works out a colour for that tab, based on whatever criteria you like. The view should contain a text calculation which outputs a colour name, one of ‘blue, yellow, green, purple, pink, turquoise, coral, lime, orange, corn’. Alternatively any of the words for colouring calculations can be used.

Then, in the manage tab of the parent table, scroll down to the tab options and select the view you created in the ‘colour from view’ dropdown.

Charting performance

Since our last update on charting improvements, many customers have been trying out more ambitious charts on large amounts of data – great to see. We have had reports of a couple of limitations so have included some updates in this release to address those:

  1. The Highcharts Boost module has been included to speed up chart rendering when there are lots of points
  2. A serverside limit has been put on the maximum number of points to avoid overloads. To create massive charts we recommend using the API to integrate with a dedicated reporting tool such as Power BI or Zoho Analytics, which some customers use to great success.

Please continue to push the boundaries and we look forward to hearing what you do.

Further adventures in charting

Earlier this month we released an update that allows the admin of a company to subscribe groups of people to charts, received by email daily, weekly or monthly.

Today’s update expands on that, adding these subscribed charts to every person’s visualisation dashboard. That means they can look at those charts in one place without waiting for a monthly (or other frequency) email.

Continuing the charting theme, another improvement is that charts now show totals for every series, which is particularly useful when charts are emailed or exported.

For users of our agile:NPDtech and agile:SA products, one of our projects on the horizon is to improve our customer success metrics, so these features will be being put to good use!

We hope these updates are useful to you immediately though. Please send any feedback to support@agilechilli.com – we look forward to hearing from you.

Reporting groups

You may know that as a user of agileBase, you can subscribe to charts, to have them emailed to you monthly, weekly or daily.

For our product agile:NPDtech we’ve recently been working on customer success metrics so that customers can keep a track of how they’re doing after onboarding, where they’re seeing benefits and what they might want to focus on. As part of that work we wanted an easy way to send these metrics to groups of people, e.g. every user who’s a director or set up as a ‘project champion’.

So when editing charts, we also added the ability for an administrator to subscribe not just themselves but any role set up in the system. All users of that role will then receive that set of charts.

We’ve found this very useful ourselves, but the feature is now available in the platform for any customer to use! We hope that you also find a use and please let us know if you’ve any feedback once you’re tried it out.

Administrator set up

As an administrator, it’s easy to set this up. First make sure a role exists in the system with the users you want to target.

Then, in the tiles view, go to a view containing charts you want them subscribed to and click the visualisations button to see charts.

Click ‘send these charts monthly’ or whichever time period you prefer, then in the roles dropdown to the right, select the role you want to be subscribed. You can repeat this to add multiple roles. There are also other options you may wish to look at, such as the ability to include a set of data with each chart.

Finally, you can include some text to be sent in the report, if you want to include some background data on the charts or explain why they’re being sent. To do this, edit the view in the admin interface, go to the manage tab and write your text in the description box.

Other news

In other news, there’s been a user interface change that lots of customers have been requesting – the ability for the system to contract the view list at the top of the tile into a set of headings. Reducing large lists of views can make the system a lot more user friendly to new users, always one of our major aims, as well as reducing wasted screen space.

Moving the mouse over the view list will cause it to expand to show every view.

Please do contact us if you’ve any feedback or ideas for future changes.

Brexit Preparations

With the Brexit deadline looming we felt the time was right to communicate what we are doing to ensure we remain compliant in case we crash out with no deal.

Perhaps the most obvious change we have needed to implement is driven by our desire to retain our EU based customers. Clearly leaving the EU doesn’t mean we no longer need to comply with EU regulations such as GDPR!

In practice, rather than just comply with the EU’s GDPR regulation themselves, we now also need to draft an additional contract per customer to say we are doing so!

Luckily the EU has created a range of “Model Clauses” for this very scenario. One these Models is specific to our situation as a SaaS provider and we can use this template, provided we make no changes.

If you are a food sector customer, to review this template please log in as normal at www.agilechilli.com and click agile:Space at the top right, to take you to the customer support portal. There, in the Documents tile, you’ll be able to download a copy, along with our GDPR and data security/privacy policies. Other customers, please contact us.

This is just one example of the changes we are expecting to need to make. We would be more than glad to hear from anyone who has any other Brexit concerns with our services and products.

Entering time durations

A very quick update on a new feature for the agileBase platform: if you have a field which is a time duration, it can now be entered in hh:mm format. This can be useful for applications like time logging.

To set this up, create a decimal number field to use as the time duration. Edit the options and tick ‘use duration’.

In the user interface, people will see an hh:mm data entry box. However when data is saved, it will be stored in the database as a decimal number representing fractions of an hour. So for example


will be transformed into 1.5 i.e. one and a half hours. That means reporting, charting and data analytics can all be used to total up time.

We hope this is useful to you and look forward to hearing your comments.

Visualising workflows

We love it when customers push the boundaries of the agileBase platform. There are always new ideas to explore.

Our latest update was inspired by a couple of customers who’ve built a lot of process automation using the agileBase workflow engine. They want to take things even further but are starting to see the need for a map of all existing workflows so they can see everything in context and check the order in which things run.

One customer has close to 150 workflow automations so being able to manage them all in one place is highly appreciated.

We’re pleased to announce a new element of the agileBase usage dashboard – Scheduled Workflows. To see this, click the bird icon at the top left of the homepage and select ‘usage dashboard’. Then just scroll down past the first few metric visualisations.

The screen shows workflows categorised by when they run – every 5 minutes, 10 minutes etc. up to daily at times you specify.

Workflow chains are marked out – that’s when a workflow has multiple linked steps, for example generating an invoice PDF then emailing it to a customer.

The icon for each workflow shows what type it is, e.g. a Chaser for sending emails and gathering feedback, a workflow to create new records or a document generator.

Workflows which are currently scheduled show a green icon – hovering over it shows the time it’s scheduled for.

Finally if any errors occur, the workflow is highlighted in red with an error description.

Clicking on any workflow will take you to it in the admin interface where you can edit its details.

Rules for deleting data

You may know that you can set up various policies for how data deletion in agileBase works:


The default and most restrictive (safest!) is that if there are any records linked to the one you’re deleting, you won’t be able to remove it until you’ve manually removed or disassociated all of the linked ones too.

That’s great for protecting important data but can make things tricky when just testing things out as a new user, demonstrating to colleagues etc., particularly when automated workflows are involved. If when you create an organisation record in a CRM system, a default site and a couple of other related bits of information are added automatically, deleting a temporary record can be quite a chore. How can we address that?

One idea we’ve introduced recently is to allow a ‘grace period’ of an hour before the more stringent deletion policies come into effect. In other words, if you create a new record, you can delete it and all it’s dependencies with one click within sixty minutes of the creation time. Only after that do the restrictions come into force.

That’s useful for users and important to know for administrators. If as an administrator you have any questions or you’d like to customise this, please get in touch with us.