agileBase Platform vision 2020 – from startup to scale-up without screwing up*

Our ambition is to create a Low Code / No Code platform that can support truly ambitious organisations to build their back office across the three phases of their development; startup, scaleup and growth.

Since agileBase’s inception we’ve had the good fortune of working with a wide range of successful customers.

Some have have been driven by dedicated individuals that have grown their companies organically, from small local businesses, through SME territory to become enterprise with hundreds of millions of pounds turnover and growing.

A fair proportion are VC-backed, started by individuals with experience in an industry, who had an idea, convinced funders of its (and their) worth and put in a lot of hard work to make the vision a reality.

Still more are long-standing family firms, where each generation brought something new to the business.

Others have grown rapidly by acquisition.

Most are increasing revenues, customer and employee numbers as they grow their businesses to cover new industries and geographies.

In fact, every story is different, though luckily success is a common factor.

What does this mean for agileBase?

We have built a platform to enable business agility through technology. Specifically, the agile development of back office systems – those that can easily grow and adapt as a company finds new challenges, or pivots to a new area, as sometimes happens.

…and where do we go from here?

When we started, we were the only platform with the technical capabilities to create advanced systems. Indeed, the reason for creating agileBase was because all the other systems we tried to use for customers had a technical ceiling of some sort that meant they had to be abandoned after a certain point in time – they couldn’t create multi-level relationships for example, or couldn’t bring data from different sources into one screen.

What now sets us apart from others who’ve entered the low-code/no-code field is the ability to scale through the entire journey of a business, whatever size they reach and however fast they move.

This is important for companies because it means that they don’t need to experience the upheaval of swapping out core systems at the very point at which they need to be focusing on their own business concerns.

Switching from one major software system to another is resource-sapping at the best of times, as anyone who’s been through the experience will testify. You don’t want to be doing that when orders, working hours and stress levels are going through the roof!

We aim to capitalise on that ability and build further on it – to prove we’re the best option for companies who think for the long term.

Where are we starting from?

We have always focussed on three things; being fast, friendly and flexible.

  • Fast to build new systems
  • Flexible enough to allow you to re-build and adapt to new circumstances just as quickly 
  • Friendly enough to be easily adopted and enjoyed by all users

We have also been very lucky in that, from the very start, we have used a rock-solid database engine that’s inherently scalable and proven at massive data volumes. That’s not something we claim to have built ourselves, but rather comes from our choice to use PostgreSQL: the world’s most advanced open source relational database.

That was an early decision that’s paid dividends ever since.

Where can we improve?

For larger, sustainable companies, additional concerns become important, such as

  • Ecosystem integration
  • Control
  • Efficiency
  • Reliability
  • Data privacy and security

What do these mean and how are we taking them into account?

Well, here is our…

Roadmap

~ Ecosystem integration ~

Over the past year, agileBase’s API has become one of the core aspects of the platform, this ‘interface to the outside world’ is as important as the user interface itself. Customers have been using it to transfer GB of data in and out of the system per month. agileBase is in effect their ‘data engine’ powering other parts of the business, from reporting systems to public websites.

Whilst it would be nice to be able to say that we foresaw all trends, truthfully some things have surprised us and the rapid increase in API use has been one. We’ve responded by putting a lot of work in to making the API interface highly scalable and performant.

In the longer term, we will be working on not just the API itself but other enterprise integration features such as single sign-on, implemented in standards compliant ways like OAuth.

~ Control ~

In the build phase, agileBase is all about agility (duh!). In large companies, we need a way to retain the holy grail of being adaptable, whilst ensuring that developed systems retain stable and trustworthy. How do we square this circle?

We’ve thought about this for a long time and have a really exciting, even revolutionary solution.

The aim is to democratise the ability to develop apps in agileBase even further.

The first thing we’re going to do is add the ability for people to modify views directly in the tiles interface, taking advantage of modern UX practices and of our recent usability learning. That will make it even simpler and quicker for non-technical users to build systems without having to flip back between an administration interface and a separate user interface. The effect will be that changes can be seen in real time as they happen.

This allows control of departmental apps to be pushed down to the departments themselves. If someone in purchasing want to tweak the PO system to update processes and create new reports, leveraging existing databases of suppliers and contacts for example, they can do, without having to go back to central IT.

However, and this is the key point, we by default hide those changes from anyone else in the system. New views that a user creates will only be visible to them. If an administrator decides they’re really useful, they can then flip a switch and make them available to everyone (with the correct permissions). So admins can mandate a core set of views that are globally available, but then let others go for it to make the most they can of their data.

~ Efficiency ~

As you know, agileBase allows applications to be built quickly, but once those applications are in use, particularly in larger organisations, they want to be scaled up so lots of users can process lots and lots of records rapidly – efficiency becomes really important.

We have one other really revolutionary change in the offing that will affect all users’ experience of agileBase. We are going to introduce a new ‘spreadsheet’ or inline editing mode so data can be edited rapidly, in bulk, without navigating back and forth from lists of data to individual records. Of course, various protections will be built in to avoid accidentally doing damage and overwriting data by mistake.

Together with the existing workflow automation features, this and other updates in the pipeline will drastically reduce the number of person-hours specific tasks take.

~ Reliability, Privacy and Security ~

It goes without saying that these are key concerns of ours, our customers and our future customers. Our GDPR policies, procedures and security features we’ve consistently built up over a long period form a bedrock upon which the exciting capabilities above can be assembled.

We will continue to follow emerging trends and implement new data security & privacy best practices.

Summary

Our core mission of ‘business agility through technology’ remains, along with our three characteristics – to be fast, friendly and flexible. They’re part of our DNA and we will always champion them. Many of the features here will be developed in the light of those.

agileBase is now being put to work in ways we didn’t originally envisage, working as the engine for scaled up business enterprises as well as startups. There are some big and exciting changes coming, do get in touch and join us for the journey.

* with apologies to Jurgen Appelo

Burndown 2019

As 2020 draws near, we’re excitedly preparing a roadmap of all of the sci-fi style features that are planned to become part of the agileBase platform.

That will be released soon. While it’s in progress, here’s a quick rundown on some interesting stats and features from the year to date. Actually, to start with, we have a long term overview. This is a chart showing the growth of the agileBase codebase over the past 10 years.

This is what’s called a ‘burndown’ chart. Each colour band represents the amount of code added in one year. The thicker the band, the more code added. After each year, the bands get thinner towards the right because the original code is modified or removed, so there’s less of it left. New code gets added ‘on top’, a bit like layers of sediment being laid down.

You may notice a large jump half-way through 2019 at the right. That corresponds with a major change to our development and release methodology. Partly, a lot of ‘testing’ code was added to allow us to run automated tests whenever a change is introduced. Also, some infrastructure was changed to let us utilise modern tools such as Gradle and CircleCI.

The outcome of those changes is that customers can better trust the reliability of the agileBase platform for running 24×7 mission critical services, even as major new features are introduced.

2019

Here are some outline stats of the other work completed since January.

For a rundown of the main changes, take a look through recent posts on https://blog.agilebase.co.uk/. Here are some of the main themes

In 2020, having laid the groundwork by making the process changes described at the top of this post, we’ll be able to make some more radical feature improvements around ensuring agileBase is the best platform for high growth businesses at allstages of their life.

Promoting Two Factor Authentication

As part of our continuing drive to improve the security of data stored in agileBase, we’re taking another step in the push to promote two factor authentication to all users.

This post is for system administrators who support agileBase users, to give you information about what’s happening. If you’d like more background information on what 2FA is and why it’s important before reading on, please see this blog post from Auth0: https://auth0.com/blog/why-every-business-needs-two-factor-authentication-security/

We introduced basic SMS based 2FA way back in 2015. Since then, app-based 2FA has been added using the industry standard Time-based One-Time Password (TOTP) algorithm, meaning that it’s compatible with the vast majority of authenticator apps out there, such as Google Authenticator, Microsoft Authenticator or Authy. This time last year, 2FA was made mandatory for anyone with system administration privileges.

We’re now trying to encourage ‘normal’ users to adopt 2FA as well as administrators and we’d really appreciate your help with this. Here’s a quick Q&A to get you up to speed:

What will users see?

Any users who haven’t yet turned on 2FA will see a prompt on the homepage once a week, on Mondays (also on weekends if they log on then). It looks like this:

How can I make things easy for users?

We’ve created a simple page and video for users to give them more information and help them set up, here: https://agilechilli.com/help-centre/agilebase/account-settings-agilebase/how-to-set-up-two-factor-authentication-2fa/

We’d love it if you can share this with your users and help them set up 2FA. If everyone activates it, it will be a big step in increasing the protection of your company data and hopefully will help your peace of mind!

Is 2FA now mandatory?

No it’s not. We would like to make this ‘on by default’ at some point in the future but we realise that it may need some extra work, for example to allow administrators to opt-out people at their discretion. Until then, users will sometimes receive a prompt when logging in but can cross it off.

What if a user has no smartphone?

If someone doesn’t have a compatible smartphone, or can’t/doesn’t want to use it for work purposes, that’s fine. You can run a 2FA app on the computer itself. One of the easiest options is probably this open source Google Chrome plugin: https://authenticator.cc/

It may even be that this is your preferred method – it can be more convenient in day to day use, though initial setup can arguably slightly more complex than taking a photo of the QR code with a phone.

What happens if a user loses or gets a new phone?

Some 2FA apps like Authy save settings online so all they need to do is re-install the app. However with others (like Google Authenticator), if you get a new phone, you have to set up 2FA again. When this happens, all the administrator has to do is un-tick ‘Enable two factor authentication’ in the user’s settings, then the user can set it up again. The user can also un-tick and re-tick this themselves to set up (as long as they can log in).

My users already use a 2FA app for other software, can they re-use that?

Yes, agileBase is compatible with the industry standard (TOTP) and all apps allow codes for multiple accounts to be added, so if a user already has an app such as Google Authenticator, Microsoft Authenticator, Duo Mobile or Authy, that can be used. Apps can also be used for multiple agileBase accounts if a user has more than one login username.

We have lots of users – how can we reduce setup time?

Since adoption is not yet mandatory, you may want to adopt a tactic such as only doing a few at a time, in cohorts for example, or training other key users in your organisation to help.

There is another option – by entering the user’s mobile phone number into their account settings, agileBase will use SMS text message codes for 2FA, so the user doesn’t have to set up 2FA themselves.

However please be aware this may result in a greater support burden yourself and potentially frustrated users. We’ve had reports that some phone networks don’t always send the codes through in a timely manner (or sometimes at all), resulting in users who are locked out. Another consideration is that text messages are less secure, vulnerable to the SIM-swap vulnerability. If it’s important to keep your company’s data safe, we strongly recommend using app-based 2FA.

Thanks for your support in helping to improve everyone’s data security.

Zapier Push Integration

We’ve long been fans of Zapier and over recent years many customers have made lots of really useful integrations. The combination of agileBase and Zapier allows you to not only develop back office systems quickly but also link them into the wider ecosystem of cloud applications, all without writing a single line of code.

We’re now improving the Zapier integration to allow even more use cases using push.

The standard way to link Zapier to agileBase is to create a Zap based on a webhook poll. Every few minutes, Zapier will poll agileBase to see if there’s any data to send over. The most frequently this can be set to happen is once every 5 minutes.

This works well for cases where the action’s not particularly time sensitive, such as sending new email addresses to Mailchimp or invoices to Xero for example. However sometimes, you (or rather the user) want the action to take place immediately, on the press of a button.

agileBase has had the facility to push data to third party systems on demand for quite a while, but up until now these haven’t worked well with Zapier, rather requiring manual integration by a coder.

Now, some options have been added to the Push API settings to work with Zapier out of the box. Furthermore, these options will default to Zapier-compatible settings for new integrations.

In Zapier, set up a webhook integration and select ‘Catch Hook’ or ‘Catch Raw Hook’.

In agileBase, paste the URL Zapier provides into the Push URL field. For Push data format, select ‘JSON content body (Zapier compatible)’ and optionally tick ‘generate simple format JSON’.

A more detailed tutorial on how to use Zapier with agileBase will be coming soon, in the meantime, please get in touch if you have any questions when you try this out.

Thanks to our partner Little House Consultancy for suggesting this improvement and our customer Lewis Pies for being the first people to beta test and try it out in anger.

Other improvements

A couple of other platform improvements have made the cut for today’s release:

Floating help panels

A great suggestion from our next-door neighbours at the Bristol Bath Science Park, Willow DNA, who are expert e-learning consultants – thanks!

When you click a question mark icon on screen to show the relevant help for a field or block, the help popup can now be dragged around the screen. You can type and use the form on screen as normal while the help remains visible. That means you don’t have to look at the help, try to remember it then close it before acting on it.

Views with charts are shown

If a view has a one or more charts, a chart icon now appears next to the view name in the menu. Hovering over the icon shows the names of all the charts.

That saves the user having to know or guess which view contains the charts they want to look at.

Customising the Interface

As a companion post to the update for users here https://blog.agilebase.co.uk/2019/10/01/big-changes/, this post for system administrators explains how to customise the new features in this release.

Navigation menu administration

As a reminder, views can be organised into groups by adding the group name into the viewname, with a dash separating them – see the last part of this blog post. If you’ve already set up groups, these will carry through into the new interface.

Any views which are not in a specific group will be placed in a new group named as per the tile.

The explanations which appear under each view name come from the view descriptions as entered in the admin interface. Go to the manage tab of a view to edit its description.

Controlling hierarchy view

The hierarchy header is displayed when a record contains a relation to another, however as an administrator you can control when this happens.

To make a relation appear as a ‘parent’ of a record, simply move it up to the top of the screen, in the fields tab of the table. Similarly, move it down to take it out of the header area. It will be displayed as a parent if it’s the first field in the table, excluding special fields: cross references, titles, images and auto-generated number sequences.

Big changes

Over the summer, we’ve been concentrating on making the agileBase user interface the best it can possibly be. We’ve run UX workshops, organised testing by people who’ve never used the system before and brainstormed a number of design ideas.

In August we introduced many small tweaks to improve the friendliness of the system. Now we’re ready to push the envelope and release two more significant updates, available from tomorrow.

Navigation menus

The first thing you may notice on logging on and opening up a tile is that we’ve improved the navigation menus near the top of the screen, making it much easier to find things. 

Clicking on one of the menu headers will open up a dropdown list of menu items underneath it. Each item consists of a view name, a count of records (if under 50) and a description of what that item contains, if necessary. That provides more context for the person in front of the screen and makes more efficient use of the space above the main contents of the page.

Clarifying Hierarchy

Data in your system is stored in what’s called a ‘relational database’, which means that records of different types can link together.

For example, a company record may have a list of contacts related to it. When editing a company, you can typically see the links to contacts in a tab, so the relationship is obvious. 

However in the contact record, it has not (up until now) been visually clear that there is a company ‘above’ the contact. In other words, that the contact is a ‘child’ record of the company.

That connection to a hierarchy has now been made obvious with the addition of a heading line at the top of a child record linking to the parent(s). Here’s an example, where you can see that the contact Kevin Williams works for Stapleton Spices.

We’ve used companies and contacts as an example, but the same thing goes for any two types of data which may be related, for example customers and orders, projects and tasks, invoices and invoice lines or staff records and training records.

Clicking the relation link (Stapleton Spices circled above) will take you directly to that record.

If the link is wrong, e.g. if Stapleton Spices has been connected to Kevin by mistake and he should actually be linked to Stapleton Herbs, you can hover the mouse over the link for a few seconds and the option to change it will appear. 

If a parent hasn’t been chosen at all, an input box will be displayed rather than a link, letting you type in or choose one.

This is a significant update to the software and we think it will make a big difference to people, helping them understand where they are in the system at a glance.

A post for system administrators will follow shortly, explaining how to customise these features in your own system. In the meantime if you’ve any questions, please do get in touch.

agileBase API and workflows growth

Recently a partner said to us ‘You know, some of your customers are actually using agilebase as an engine to power their business, in ways that go beyond simply building applications.’  

And it’s true, agileBase is being used not just to deliver user-facing apps, but to act as a central store, processing data which is fed in and out of multiple third party systems, including for transaction processing, interacting with machinery and feeding public websites. It’s a service upon which other elements can build.

Emergent use-cases like these occur so often with startups: customers find new and exciting ways to utilise the technology which the founders didn’t necessarily envisage. This in turn opens up opportunities to grow the business in new ways and to provide way more value than ever before.

This use does however place more demands on people and systems, covering many facets, including

  • Scale: the throughput of data possible with API-based inflows and outflows is much greater than that of human beings manually interacting with systems. The same goes for automated workflows which can act on many thousands of records per second
  • Criticality: if a user can’t do their job for 5 minutes because the system is down or broken, that can be critical, but if an interaction with a third party system fails for 5 minutes causing multitudes of transactions to fail and potentially reputational damage, that’s much more likely to be critical. Both of these events have very low probability but still the consequence of the latter is far greater.
  • Complexity: when agileBase transitions from a platform powering user-facing apps to an engine powering a network of interdependent and often third party services, system complexity increases, so new means of dealing with this need to surface. 

We recognise this is a major change. If growth in this area continues at its current trajectory and nothing is done, the system will start to be overwhelmed, with all customers being affected.

However, that’s not going to happen as we’re not standing still. We’ve already started implementing concrete changes to ensure we’re ahead of the curve. We’re serious about planning for and supporting this use of the platform, as opposed to just capping usage at a certain number of requests per minute or limiting complexity artificially.

Work we’ve started

At a board level we have agreed to commit to support the option for customers to use “agileBase as an engine”, in other words following the principles behind Service Oriented Architectures.

As a direct consequence of this we have engaged consultants to help us identify how we need to adapt, both technically and commercially. 

Technical – platform development

Some immediate recommendations we’ve actioned or planned are

  • Employing people to increase the test coverage of
    • APIs 
    • Workflows 
    • Interfaces 

all of which must, given the increased criticality of the use case, have a higher level of stability and predictability. We’re also

  • Updating development processes to increase automation and reduce errors. (Amongst the tools in our armory are continuous integration, unit tests, static analysis and 3rd party library auditing)
  • Updating deployment processes to minimise downtime
  • Taking measures to shape API traffic, to avoid usage spikes from one source impacting the whole system
  • Introducing new workflow features (see below) to make them even more powerful and useful for automations

In the longer term, we’ve a roadmap which goes much further, with items such as

  • completely removing the need for any downtime at all when deploying new versions
  • scaling horizontally for virtually unlimited capacity increases

Commercial changes

Customers will now be able to run a much higher volume of workflows and rely on larger volumes of transactions flowing through the API, which opens up new ways of using the platform.

We will in future be introducing two new pricing elements to accommodate this, one for workflows and the other for APIs. These will allow us to

  • ensure the platform can move forward at the required pace
  • properly service and support customer requirements in the professional ways expected of us. 

Please be assured that general base costs will not be rising, only customers using these particular features will need to factor them in.

We greatly appreciate the innovation that current customers have brought to the table by using the system in this novel way. It’s what helps push us, and everyone else, forward. 

Therefore we want to recognise this value and help these companies by offering a free remote session or face to face meeting, to

  1. Uncover your most important mission critical elements so we can, at a later date, build customer-specific unit tests to cover these elements to add to our growing test suite
  2. Gather further feedback and requirements about what would make the system even more useful, with the results being prioritised in our roadmap

We will be getting in touch with any customers who may benefit from this, and to assess whether there are any cost implications for the future (there will be no immediate changes). 

As always, we look forward to talking to you. Get in touch by any of the methods listed on our contact page, or drop in and visit us at Bristol & Bath Science Park.

Background to Workflows and APIs

Fundamental to many of these new and innovative usage patterns are two features – workflows and APIs.

The thinking behind the development of these features followed the same pattern we’ve used while creating the whole agileBase platform; make things which are powerful because they’re easy to use (following common principles) and flexible i.e. generic enough to be used for many different purposes.

For the technically minded, APIs can be auto-generated for any View in the system, by ticking a box. Developers get an industry standard Restful, JSON based API based on the contents of the view, complete with https://swagger.io/ description for easy import to third party systems.

Similarly, workflows are built on Views. Using the built in agileBot you can set rules for how to automatically process data – whether creating, updating or deleting it, as well as generating documents and automating email communications to and from people.

Example: one customer running a membership franchise uses workflows to automatically recognise and de-dupe leads coming from multiple sources, to aid the attribution of leads back to marketing campaigns. APIs then send and receive data from the website to track customers’ touch points and spending.

To help customers with this particular emergent usage pattern, we’ll be introducing enhancements to both of these features, so watch this space.