Zapier Push Integration

We’ve long been fans of Zapier and over recent years many customers have made lots of really useful integrations. The combination of agileBase and Zapier allows you to not only develop back office systems quickly but also link them into the wider ecosystem of cloud applications, all without writing a single line of code.

We’re now improving the Zapier integration to allow even more use cases using push.

The standard way to link Zapier to agileBase is to create a Zap based on a webhook poll. Every few minutes, Zapier will poll agileBase to see if there’s any data to send over. The most frequently this can be set to happen is once every 5 minutes.

This works well for cases where the action’s not particularly time sensitive, such as sending new email addresses to Mailchimp or invoices to Xero for example. However sometimes, you (or rather the user) want the action to take place immediately, on the press of a button.

agileBase has had the facility to push data to third party systems on demand for quite a while, but up until now these haven’t worked well with Zapier, rather requiring manual integration by a coder.

Now, some options have been added to the Push API settings to work with Zapier out of the box. Furthermore, these options will default to Zapier-compatible settings for new integrations.

In Zapier, set up a webhook integration and select ‘Catch Hook’ or ‘Catch Raw Hook’.

In agileBase, paste the URL Zapier provides into the Push URL field. For Push data format, select ‘JSON content body (Zapier compatible)’ and optionally tick ‘generate simple format JSON’.

A more detailed tutorial on how to use Zapier with agileBase will be coming soon, in the meantime, please get in touch if you have any questions when you try this out.

Thanks to our partner Little House Consultancy for suggesting this improvement and our customer Lewis Pies for being the first people to beta test and try it out in anger.

Other improvements

A couple of other platform improvements have made the cut for today’s release:

Floating help panels

A great suggestion from our next-door neighbours at the Bristol Bath Science Park, Willow DNA, who are expert e-learning consultants – thanks!

When you click a question mark icon on screen to show the relevant help for a field or block, the help popup can now be dragged around the screen. You can type and use the form on screen as normal while the help remains visible. That means you don’t have to look at the help, try to remember it then close it before acting on it.

Views with charts are shown

If a view has a one or more charts, a chart icon now appears next to the view name in the menu. Hovering over the icon shows the names of all the charts.

That saves the user having to know or guess which view contains the charts they want to look at.

Customising the Interface

As a companion post to the update for users here, this post for system administrators explains how to customise the new features in this release.

Navigation menu administration

As a reminder, views can be organised into groups by adding the group name into the viewname, with a dash separating them – see the last part of this blog post. If you’ve already set up groups, these will carry through into the new interface.

Any views which are not in a specific group will be placed in a new group named as per the tile.

The explanations which appear under each view name come from the view descriptions as entered in the admin interface. Go to the manage tab of a view to edit its description.

Controlling hierarchy view

The hierarchy header is displayed when a record contains a relation to another, however as an administrator you can control when this happens.

To make a relation appear as a ‘parent’ of a record, simply move it up to the top of the screen, in the fields tab of the table. Similarly, move it down to take it out of the header area. It will be displayed as a parent if it’s the first field in the table, excluding special fields: cross references, titles, images and auto-generated number sequences.

Big changes

Over the summer, we’ve been concentrating on making the agileBase user interface the best it can possibly be. We’ve run UX workshops, organised testing by people who’ve never used the system before and brainstormed a number of design ideas.

In August we introduced many small tweaks to improve the friendliness of the system. Now we’re ready to push the envelope and release two more significant updates, available from tomorrow.

Navigation menus

The first thing you may notice on logging on and opening up a tile is that we’ve improved the navigation menus near the top of the screen, making it much easier to find things. 

Clicking on one of the menu headers will open up a dropdown list of menu items underneath it. Each item consists of a view name, a count of records (if under 50) and a description of what that item contains, if necessary. That provides more context for the person in front of the screen and makes more efficient use of the space above the main contents of the page.

Clarifying Hierarchy

Data in your system is stored in what’s called a ‘relational database’, which means that records of different types can link together.

For example, a company record may have a list of contacts related to it. When editing a company, you can typically see the links to contacts in a tab, so the relationship is obvious. 

However in the contact record, it has not (up until now) been visually clear that there is a company ‘above’ the contact. In other words, that the contact is a ‘child’ record of the company.

That connection to a hierarchy has now been made obvious with the addition of a heading line at the top of a child record linking to the parent(s). Here’s an example, where you can see that the contact Kevin Williams works for Stapleton Spices.

We’ve used companies and contacts as an example, but the same thing goes for any two types of data which may be related, for example customers and orders, projects and tasks, invoices and invoice lines or staff records and training records.

Clicking the relation link (Stapleton Spices circled above) will take you directly to that record.

If the link is wrong, e.g. if Stapleton Spices has been connected to Kevin by mistake and he should actually be linked to Stapleton Herbs, you can hover the mouse over the link for a few seconds and the option to change it will appear. 

If a parent hasn’t been chosen at all, an input box will be displayed rather than a link, letting you type in or choose one.

This is a significant update to the software and we think it will make a big difference to people, helping them understand where they are in the system at a glance.

A post for system administrators will follow shortly, explaining how to customise these features in your own system. In the meantime if you’ve any questions, please do get in touch.

agileBase API and workflows growth

Recently a partner said to us ‘You know, some of your customers are actually using agilebase as an engine to power their business, in ways that go beyond simply building applications.’  

And it’s true, agileBase is being used not just to deliver user-facing apps, but to act as a central store, processing data which is fed in and out of multiple third party systems, including for transaction processing, interacting with machinery and feeding public websites. It’s a service upon which other elements can build.

Emergent use-cases like these occur so often with startups: customers find new and exciting ways to utilise the technology which the founders didn’t necessarily envisage. This in turn opens up opportunities to grow the business in new ways and to provide way more value than ever before.

This use does however place more demands on people and systems, covering many facets, including

  • Scale: the throughput of data possible with API-based inflows and outflows is much greater than that of human beings manually interacting with systems. The same goes for automated workflows which can act on many thousands of records per second
  • Criticality: if a user can’t do their job for 5 minutes because the system is down or broken, that can be critical, but if an interaction with a third party system fails for 5 minutes causing multitudes of transactions to fail and potentially reputational damage, that’s much more likely to be critical. Both of these events have very low probability but still the consequence of the latter is far greater.
  • Complexity: when agileBase transitions from a platform powering user-facing apps to an engine powering a network of interdependent and often third party services, system complexity increases, so new means of dealing with this need to surface. 

We recognise this is a major change. If growth in this area continues at its current trajectory and nothing is done, the system will start to be overwhelmed, with all customers being affected.

However, that’s not going to happen as we’re not standing still. We’ve already started implementing concrete changes to ensure we’re ahead of the curve. We’re serious about planning for and supporting this use of the platform, as opposed to just capping usage at a certain number of requests per minute or limiting complexity artificially.

Work we’ve started

At a board level we have agreed to commit to support the option for customers to use “agileBase as an engine”, in other words following the principles behind Service Oriented Architectures.

As a direct consequence of this we have engaged consultants to help us identify how we need to adapt, both technically and commercially. 

Technical – platform development

Some immediate recommendations we’ve actioned or planned are

  • Employing people to increase the test coverage of
    • APIs 
    • Workflows 
    • Interfaces 

all of which must, given the increased criticality of the use case, have a higher level of stability and predictability. We’re also

  • Updating development processes to increase automation and reduce errors. (Amongst the tools in our armory are continuous integration, unit tests, static analysis and 3rd party library auditing)
  • Updating deployment processes to minimise downtime
  • Taking measures to shape API traffic, to avoid usage spikes from one source impacting the whole system
  • Introducing new workflow features (see below) to make them even more powerful and useful for automations

In the longer term, we’ve a roadmap which goes much further, with items such as

  • completely removing the need for any downtime at all when deploying new versions
  • scaling horizontally for virtually unlimited capacity increases

Commercial changes

Customers will now be able to run a much higher volume of workflows and rely on larger volumes of transactions flowing through the API, which opens up new ways of using the platform.

We will in future be introducing two new pricing elements to accommodate this, one for workflows and the other for APIs. These will allow us to

  • ensure the platform can move forward at the required pace
  • properly service and support customer requirements in the professional ways expected of us. 

Please be assured that general base costs will not be rising, only customers using these particular features will need to factor them in.

We greatly appreciate the innovation that current customers have brought to the table by using the system in this novel way. It’s what helps push us, and everyone else, forward. 

Therefore we want to recognise this value and help these companies by offering a free remote session or face to face meeting, to

  1. Uncover your most important mission critical elements so we can, at a later date, build customer-specific unit tests to cover these elements to add to our growing test suite
  2. Gather further feedback and requirements about what would make the system even more useful, with the results being prioritised in our roadmap

We will be getting in touch with any customers who may benefit from this, and to assess whether there are any cost implications for the future (there will be no immediate changes). 

As always, we look forward to talking to you. Get in touch by any of the methods listed on our contact page, or drop in and visit us at Bristol & Bath Science Park.

Background to Workflows and APIs

Fundamental to many of these new and innovative usage patterns are two features – workflows and APIs.

The thinking behind the development of these features followed the same pattern we’ve used while creating the whole agileBase platform; make things which are powerful because they’re easy to use (following common principles) and flexible i.e. generic enough to be used for many different purposes.

For the technically minded, APIs can be auto-generated for any View in the system, by ticking a box. Developers get an industry standard Restful, JSON based API based on the contents of the view, complete with description for easy import to third party systems.

Similarly, workflows are built on Views. Using the built in agileBot you can set rules for how to automatically process data – whether creating, updating or deleting it, as well as generating documents and automating email communications to and from people.

Example: one customer running a membership franchise uses workflows to automatically recognise and de-dupe leads coming from multiple sources, to aid the attribution of leads back to marketing campaigns. APIs then send and receive data from the website to track customers’ touch points and spending.

To help customers with this particular emergent usage pattern, we’ll be introducing enhancements to both of these features, so watch this space.

Upcoming release

Today we’re taking the unusual step of announcing details of an upcoming release in advance. It’s a feature packed release with some significant updates.

Homepage makeover

The first thing you’ll notice is the new icons! We’re using Duotone, a beautiful new icon style available as part of Font Awesome Pro. As well as the stylish new look, there are more icons to choose from for app developers. Our favourite is ‘pepper-hot’:

We have also changed the way the Tiles on the Home Page display to give you more control. The order can now be fixed by by going into the “Tiles” area of the administration interface and using the numbering.

Tiles will still change size depending on how often they’re opened, but the least used won’t move to the end of the screen. It was a nice idea but actually, testing has shown that the system’s easier to use when they remain in the same positions. Maintaining a logical order is now possible, so you can put suppliers next to customers or ingredients before products.

User interface tweaks

Many other elements of the system have been tweaked, following analysis of our most recent UX testing programme. For example, number prefixes in dropdowns have been removed (the numbering is used purely for ordering purposes) and there’s a new UI to show sub-tabs.

Even more UI changes are planned, so watch this space!

Auto-updating calculations

When using inline editing in a tab, e.g. as in the screenshot below of editing a recipe, all calculations will update as you type. In this example when you change an ingredient quantity, all the percentages update as well as the quantity total at the bottom of the list.

On demand workflows

Until now you’ve been able to set up automated workflows to run regularly, up to every 5 minutes, or on creation of a new record. Now you can also run them every time a record is loaded by a user. Since you can do so much with workflows, like edit records based on rules, or generate documents, this gives you the ability to set up automations to work in a much more user friendly way, without people having to refresh records or wait for changes.

As we say, there’s lots more in the pipeline and we’ll shortly have some news to share about a new type of usage pattern for agileBase. In the meantime, we hope you enjoy the new user interface which will be released in the next few days – we always aim to keep the system fast, friendly and flexible.

Export controls

Near the end of last year, we introduced export notifications for sensitive commercial or personal data. Along with various other features, that lets administrators of a system be notified by email if a user exports more than a certain amount of data.

Well, this week’s update expands on that. As well as being notified, administrators can, per table, control how much a user can export at any one time, i.e. they can set a hard limit. This limit takes effect per export, so bear in mind it wouldn’t stop someone exporting an entire database of contacts for example, by doing lots of different filters and exports one at a time (e.g. all As, all Bs, all Cs etc.) but it would increase the effort and the administrator would get lots of notifications.

Remember, you can also set privileges to prevent any exporting at all for certain groups of people and/or views.


To set this up, go to the Build section in the agileBase admin interface. Then go to the manage tab for the table you’d like to control.

Make sure that one or both of ‘stores sensitive’ or ‘stores personal’ data are ticked and you’ll see two options. The first lets you set the limit at which admins will be notified of exports (e.g. 10 records). The second, which is the new option, lets you set the ‘hard’ limit. If someone tries to export more than this, the rows exported will automatically be cut off after this number.

The export limit applies to all views based on the table, in fact all views which any data from the table appears in. If a view uses data from many different tables, the minimum export limit will be applied when exporting from that view.

Please take a moment to think whether this will be useful for your organisation and set it up if so – we want all customer data to be as well protected as possible.

If you’ve any queries, don’t hesitate to get in touch as usual.

Big data

Last week we started talking about scalability in terms of the API and that got me thinking about what we’d do if and when we need to process orders of magnitude more data than we do currently.

Currently, the largest single table of data any one customer has comprises just over thirty million records. That may be a lot of business data in some contexts but in database terms it’s hardly ‘big data’. What about if we needed to store and work with billions of records?

A natural direction to look in would be to As you may know, agileBase uses the open source PostgreSQL database. Citus transforms PostgreSQL into a distributed database. And it’s open source too. Their pitch is ‘Never worry about scaling again’.

Citus has recently been bought by Microsoft and is now available as Hyperscale (Citus), a built-in deployment option for Azure Database for PostgreSQL. It can also be used on AWS or your own hosting. So there are many options to allow scaling to many nodes to handle billions or hundreds of billions of rows, while maintaining high performance.

That’s for the future, but if anyone has an inkling of any projects they’d like to put forward which deal in high volumes of data a.k.a. ‘big data’ in the IT world, then drop us a line. We’d be keen to be involved.

Updates roundup

It’s been a while since the last product update on the 1st April, but that doesn’t mean we haven’t been hard at work developing the agileBase platform further. It just means we’ve been too busy working with new customers to do much blogging (plus taking the odd summer holiday to be fair).

However, there are some great new features recently released to tell you about, so let’s summarise them quickly:

WYSIWYG editing

This one had been on the roadmap for a while and some specific use cases from new customers pushed it onto the active pile. In short, when editing text in a large text box in the user interface, people can now easily format their content, adding bold, underline and italics for example.

To format a word, just double click it, or use keyboard shortcuts like Ctrl+B / Cmd+B to make text bold, as in any common wordprocessor. Here’s what it looks like in use:

Image extractor

This is one for the ‘experimental’ list – we’re releasing it as a beta feature and hopefully it’s really useful for some people, but bear in mind that beta label.

If you have an image field with type ‘identifier image’ in a table, when you type a web address into a text field into the same table, agileBase will visit that website to extract and store what it thinks is the main image on it – most often a banner image near the top, or possibly a logo. That can add a bit of interest if you have company information in a CRM for example. Of course, the system may not always be able to get the best image, or any image so you can always manually upload.

If you’d like to try this out, please let us know.

API scalability

We keep a keen eye on the performance and scalability of our services, starting with efficiency at the lowest level of algorithms and programming – after all if a single thread or server is efficient, that also makes adding more hardware (scaling ‘horizontally’ or ‘vertically’) much more effective.

However, most of this work to date has focussed at the level of the database or our own code. Many newer customers are heavily using agileBase not just to build business apps in their own right, but more of a data engine to power a whole host of inter-related infrastructure, for example transactional websites, mobile apps and feeding into enterprise-wide reporting.

All of this relies on the API – the mechanism by which agileBase communicates with other software. The long and the short of it is we’ve made some major improvements to increase the volume of data that can flow in and out without impacting other operations, such as making changes to the system at the same time, like adding a table or field for example.

We sat down with an expert (thanks Rich) to work out how we could increase scalability in this area. Some ideas have been actioned and some are in the bag to pull out in future as and when needed.

We hope these updates are useful to current and indeed future customers. Please feel free to contact us to get involved with the continuing process of improving the agileBase platform!

Visual refresh

For the start of April, we have a Spring refresh of the record editing interface for agileBase, making forms more friendly and breathing new life into the interface.

Continuing the previous work on form simplification such as ‘contracting blocks‘, this makes working in apps built on the agileBase platform an even more pleasant experience. There are various improvements to style and alignments, the most obvious being that form field outlines are changed from boxes to underlines, to reduce visual clutter.

Tab colouring

In addition, administrators have a new UI tool in their armoury – coloured tabs. Tabs on screen can now be colour coded, to draw users attention or give them information. For example, if a complex process required half a dozen tabs, you may want to colour them green as each step is completed.

Setting this up works on views, in a similar way to many other features in agileBase. From the parent table of the tab, create a view which works out a colour for that tab, based on whatever criteria you like. The view should contain a text calculation which outputs a colour name, one of ‘blue, yellow, green, purple, pink, turquoise, coral, lime, orange, corn’. Alternatively any of the words for colouring calculations can be used.

Then, in the manage tab of the parent table, scroll down to the tab options and select the view you created in the ‘colour from view’ dropdown.

Charting performance

Since our last update on charting improvements, many customers have been trying out more ambitious charts on large amounts of data – great to see. We have had reports of a couple of limitations so have included some updates in this release to address those:

  1. The Highcharts Boost module has been included to speed up chart rendering when there are lots of points
  2. A serverside limit has been put on the maximum number of points to avoid overloads. To create massive charts we recommend using the API to integrate with a dedicated reporting tool such as Power BI or Zoho Analytics, which some customers use to great success.

Please continue to push the boundaries and we look forward to hearing what you do.

Further adventures in charting

Earlier this month we released an update that allows the admin of a company to subscribe groups of people to charts, received by email daily, weekly or monthly.

Today’s update expands on that, adding these subscribed charts to every person’s visualisation dashboard. That means they can look at those charts in one place without waiting for a monthly (or other frequency) email.

Continuing the charting theme, another improvement is that charts now show totals for every series, which is particularly useful when charts are emailed or exported.

For users of our agile:NPDtech and agile:SA products, one of our projects on the horizon is to improve our customer success metrics, so these features will be being put to good use!

We hope these updates are useful to you immediately though. Please send any feedback to – we look forward to hearing from you.