Roundup – What’s New

What’s new in the agileBase platform this week? As usual, we haven’t been sitting on our hands and there are some interesting new updates.

Font Awesome

The icons used in agileBase, as part of the user interface and from which an administrator to represent each tile, have been upgraded to Font Awesome version 5, bringing a massive number of extra icons and a fresh, clean look, as well as some usability enhancements – ticked checkboxes are easier to differentiate.

We are a proud funder of the Font Awesome 5 Kickstarter, the most funded software Kickstarter of all time. As such we’re able to make over a thousand icons available in agileBase including the Pro pack – whatever module or app you want to build, there’ll be an icon for it!

Multi-tenanting

Some customers use the multi-tenanting capabilities of agileBase – the ability to separate users into groups who can’t see each other’s data. This can be used for example to allow different sets of companies, clubs, or membership groups (or whatever the organisational group is) to use the same system features whilst remaining separate from each other.

To recap, the way this is set up is that a role is created for each group with a ‘filtering’ field name and value. Whenever a view contains a field with that name, each user will only see rows where the value matches the value of the role they’re in. If a view doesn’t contain that field, data is visible to all users, which may be what’s required in some situations, for example where there’s a shared dataset such as a collection of help articles.

Now, views without a filtering field will be highlighted in the admin interface in red. So an admin can see at a glance which views will be filtered, which should help stop any from falling through the cracks.

Multi-tenanted Audit Trails

A second enhancement to do with multi-tenanting is that all audit trail logs now include the multi-tenanting value for the role a user’s in. That means an administrator can easily compare usage for each group, or even give access to the audit trails to each group so they can see their own use.

Advertisements

Revolutionary Speed Increases

The agileBase strapline is ‘fast, friendly, flexible’. By ‘fast’ we usually mean fast to develop and prototype, i.e. you can typically get working business databases built and running in minutes or hours.

However, it has another meaning too – we’ve always strived to have the fastest response times as users interact with apps built on the platform. That’s down the the many optimisations we build in to both the server and user interface – too many to list here – as well as the choice of technology (props to PostgreSQL, the world’s most advanced open source database) and high performance hosting (thanks Linode).

As the number of customers grows and the sizes of their systems balloon, we need to always bear performance in mind.

Today we’re excited to announce the most effective single change we’ve ever made to improve performance – the ability to use caching on high transaction rate views.

Caching in general has been an option for a while (materialization in database terminology). When you have complex views that are slow to query, but don’t update that much, it’s great. A common example would be a financial report totalling up sales per month. There may be millions of rows of data involved. Caching can make querying, filtering and charting much more responsive, so data loads in milliseconds rather than seconds.

To date, the options have been to update the cache every 10 minutes or once a day, which is fine for those sorts of views where new data isn’t added that often.

However, what about views which are complex, perhaps with many calculations and lots of data, but which are also updated and used very frequently? Stock figures, sales jobs or orders for example. These are the ones which have most impact on people in day to day use, and on the system in general, which affects everyone.

The system will now update the cache whenever someone saves a new record, updates one or deletes one. This makes querying the data very fast. Typically, there are a lot more queries of data than there are edits, so the performance of the whole system is improved. For further efficiency, not every edit triggers a cache update, all changes are rolled up into one when the user moves from the editing screen back to the list of records.

What delay there is is also psychologically and practically in the right place. When users first open a tile to look at data, they want the response to be immediate. It’s important that searching for data is also very rapid. However, a short delay just after pressing save can be tolerable – the user has just finished doing something, as opposed to starting it. And of course the UI reacts immediately to let them know the system is ‘saving’.

Google has some good notes on the user perception of performance delays. As they say,

0 to 100ms: Respond to user actions within this time window and users feel like the result is immediate. Any longer, and the connection between action and reaction is broken.

We try to keep the data querying actions within this timeframe. Then

300 to 1000ms: Within this window, things feel part of a natural and continuous progression of tasks. For most users on the web, loading pages or changing views represents a task.

Reloading the list of data after a record save falls into this category.

fast-1

Setting this up

As an administrator, if you have a view you’d like to speed up, how do you do so?

In the admin interface, just go to the manage tab for the view, click ‘advanced options’ and under ‘cache view rows’, select ‘cache view, update on record save’.

Welcome to a faster agileBase!

Mandating Two Factor Authentication

We’ve had two-factor authentication (2FA) capabilities for around three years now in the agileBase platform. Along with the other platform features, we’ve been keeping it up to date and improving as time goes on.

For example, as of a few months ago you can set up app-based 2FA for enhanced robustness.

Two factor authentication is now widely used in the world of software and generally understood by users, so we can start promoting it’s use a bit more. At the end of the day it is a practical security mechanism that for little effort can greatly reduce the chances of accounts being hacked. Everyone’s usually a fan of that. Find out more.

So the agileBase platform now requires that the most privileged actions require 2FA to be enabled. Those are

  1. Giving a user administrator privileges on the whole account
  2. Giving user manage privileges on a table

An administrator can effectively do anything within the customer account – add/remove users, assign roles and privileges and create/destroy any tables, views or data.

Someone with manage privileges on a table can make more limited changes to that table only but can still remove fields and delete/edit all data in bulk.

When an administrator assigns those privileges to someone, the system checks that the user has 2FA enabled first. If privileges are assigned to a role, everyone in that role must have 2FA enabled.

This is a step along the road to promoting 2FA throughout the user base. Please let us know if you’ve any thoughts or suggestions.

Calendar Timelines for Projects

A while ago we introduced the timeline feature, which comes with day grid, week grid, month grid and year grid views, to add to the standard day, week and month views.

This allows you to see a list of views down the lefthand side of the screen (e.g. audits, due, meetings, expiring certificates or whatever you keep track of) and the events flowing in date order from left to right across the screen.

Our new feature released for beta testing today allows you to split out events from a single view into different tracks down the lefthand side of the screen. In other words, instead of having to create a different view for each track, which could be cumbersome if there are many, you can just specify a field to generate the tracks from.

For example, if we have a view of project milestones like this New Product Development tracker:

milestons list view

we can choose to split on the ‘name’ field (Beef Pie is shown above), resulting in a calendar grid view like this:

milestones calendar view

Now it’s easy to keep track of the various stages of multiple projects on one screen, to see when you may need to requisition more resources or move things around.

In this example we’ve also used the colouring feature introduced recently – those colours are picked up in the calendar grid view.

This feature will be working it’s way into our new Stage Gates module for our agile:NPDtech product coming soon but it’s also available now for any customer to use for their own bespoke systems, at no charge while being beta tested. As a new feature in testing, please let us know what you think, including any glitches you come across or improvements you can think of.

Administrator set up

As an administrator, enabling this is easy – create a text calculation in the view you want to use called ‘calendar splitter’. Make the output of this the items you want to create calendar tracks for. In the example above for example, we just created the calculation as {product name}. You can then mark the calculation as hidden if users don’t need to see it.

Then add that view to your calendar.

We hope you find it useful and please do let us know any other applications you think of.

 

 

Hello Autumn

autumn

As the leaves start to turn, we’ve selected a range of login screens to match the changing season. As usual, Gemma has curated a set of stunning photos that will brighten up everyone’s morning as they log in.

For all of the agile:NPDtech customers using the agileBase platform, some of the food-based screens may even give you ideas about new products! And for everyone else, maybe they’ll just give you ideas about what to have for supper.

We hope you have an enjoyable and profitable Autumn.

User controlled caching

One of the most effective features we’ve added to the agileBase platform to improve performance is the ability to cache views. That can dramatically speed up views which do lots of number crunching.

A typical example might be something like looking at a product list report which contains all the stats about sales per year, costs, average customer feedback etc. If those calculations look at millions of data points, the view may take a good few seconds to load, but when cached, can be accessed immediately, for sorting, filtering and charting, which makes the user experience a whole lot more friendly.

The only potential downside is that until now there’s been no way for the user to see how fresh the data is. For example if a new product is added it won’t show up in that view until the cache is refreshed, which could be from 10 minutes until up to a day, depending on how the administrator’s set it up.

Now, a message appears at the top of a cached view, showing when it was last cached, when it’s next due to be auto-cached and most usefully, a button allowing the user to manually refresh the data whenever they need to.

cache refresh

 

If you’re looking at a view which you know is cached but there’s no message there, don’t worry, that’s because it’s only shown when data in the view may have changed since the last refresh time, which the system can detect.

As always, please let us know any feedback you have so we can carry on improving the system in the best way possible.

 

API growth

The ability to interoperate and communicate with other software is fundamental to the value a cloud system can offer, which is why most providers, including ourselves, use APIs. These allow different systems to talk to each other using a common language.

api
API calls – image by Tsahi Levent-Levi

There is normally some means of controlling the number of data transfers, otherwise a small (or indeed large) number of customers could accidentally swamp the system with a massive number of requests, negatively affecting themselves and everyone else. Here’s an example of how one service applies bandwidth management to avoid this:

https://api.stackexchange.com/docs/throttle

With the agileBase platform, communicating with third party software is really easy – custom APIs can be enabled by an administrator with one click for each view and tools such as Zapier then allow you to send and receive data from common cloud software tools with no need for programming.

That’s led to an increase in the number of customers using the API and a ballooning of what they’re using it for. We love seeing that, but it does mean that we now also need to think about how to protect customers against the possibilities of mis-configurations in third party systems that perhaps make them ‘overly chatty’. We need a way to smooth out demand.

For example, we’ve had one or two situations where the system has been flooded with requests from a third party due to a bug in their system that caused repeated requests for the same information in a continuous loop. Say for example a request takes on average 1 second to process and the system receives 2 requests per second. Well that means that the number of requests being actively processed at one time is pretty quickly going to spin out of control!

When that happens, people using the data being requested will of course start to experience a slowdown and lack of responsiveness. If it goes on for long enough, all other customers and users will also be affected. As a customer, you obviously don’t want your service to suffer due to events unrelated to your use and outside your control. We’ve often been praised by users for the responsiveness and speed of the agileBase platform and we’re keen to keep it that way!

Smoothing data flows

To help avoid this possibility, we now employ a couple of simple bandwidth management measures.

  1. Queuing – API requests are queued for each customer. This means that only one request can be processed at a time, to stop lots of requests coming in at exactly the same moment, which can happen just by chance if they’re all coming from different sources.
    Each customer has their own queue, so one customer’s requests won’t affect another’s.
  2. Traffic shaping – after a request for a large amount of data, the system will pause for a short time before processing further requests for a customer. The period is related to the number of records requested. 1000 records will cause a delay of 0.5 seconds, topping out at 5 seconds for 10,000 or more records.

Additionally, for particularly intensive API calls which may have a detrimental effect on other users if used too frequently, we have the option, in consultation with customers, of rate limiting. That means setting a maximum frequency for calls, e.g. once every minute, 10 minutes or hour. In fact, if you like, you can set this yourself, to protect you from third party apps you may use overloading your systems. The option is available in the API settings ‘sync’ screen under the manage tab for a view.

With these measures in place, we’re confident that businesses can take advantage of the possibilities opened up by allowing easy communication with other cloud software, without needing to worry about whether there may be any detrimental performance effects. We encourage people to look into the possibilities of setting up syncs like these examples from customers:

  • exporting data to Power BI to allow consolidated reporting from many sources
  • transferring invoices and other financial data to an accounting package like Xero
  • importing enquiries and orders from websites