Last week we started talking about scalability in terms of the API and that got me thinking about what we’d do if and when we need to process orders of magnitude more data than we do currently.
Currently, the largest single table of data any one customer has comprises just over thirty million records. That may be a lot of business data in some contexts but in database terms it’s hardly ‘big data’. What about if we needed to store and work with billions of records?
A natural direction to look in would be to www.citusdata.com. As you may know, agileBase uses the open source PostgreSQL database. Citus transforms PostgreSQL into a distributed database. And it’s open source too. Their pitch is ‘Never worry about scaling again’.
Citus has recently been bought by Microsoft and is now available as Hyperscale (Citus), a built-in deployment option for Azure Database for PostgreSQL. It can also be used on AWS or your own hosting. So there are many options to allow scaling to many nodes to handle billions or hundreds of billions of rows, while maintaining high performance.
That’s for the future, but if anyone has an inkling of any projects they’d like to put forward which deal in high volumes of data a.k.a. ‘big data’ in the IT world, then drop us a line. We’d be keen to be involved.
It’s been a while since the last product update on the 1st April, but that doesn’t mean we haven’t been hard at work developing the agileBase platform further. It just means we’ve been too busy working with new customers to do much blogging (plus taking the odd summer holiday to be fair).
However, there are some great new features recently released to tell you about, so let’s summarise them quickly:
This one had been on the roadmap for a while and some specific use cases from new customers pushed it onto the active pile. In short, when editing text in a large text box in the user interface, people can now easily format their content, adding bold, underline and italics for example.
To format a word, just double click it, or use keyboard shortcuts like Ctrl+B / Cmd+B to make text bold, as in any common wordprocessor. Here’s what it looks like in use:
This is one for the ‘experimental’ list – we’re releasing it as a beta feature and hopefully it’s really useful for some people, but bear in mind that beta label.
If you have an image field with type ‘identifier image’ in a table, when you type a web address into a text field into the same table, agileBase will visit that website to extract and store what it thinks is the main image on it – most often a banner image near the top, or possibly a logo. That can add a bit of interest if you have company information in a CRM for example. Of course, the system may not always be able to get the best image, or any image so you can always manually upload.
If you’d like to try this out, please let us know.
We keep a keen eye on the performance and scalability of our services, starting with efficiency at the lowest level of algorithms and programming – after all if a single thread or server is efficient, that also makes adding more hardware (scaling ‘horizontally’ or ‘vertically’) much more effective.
However, most of this work to date has focussed at the level of the database or our own code. Many newer customers are heavily using agileBase not just to build business apps in their own right, but more of a data engine to power a whole host of inter-related infrastructure, for example transactional websites, mobile apps and feeding into enterprise-wide reporting.
All of this relies on the API – the mechanism by which agileBase communicates with other software. The long and the short of it is we’ve made some major improvements to increase the volume of data that can flow in and out without impacting other operations, such as making changes to the system at the same time, like adding a table or field for example.
We sat down with an expert (thanks Rich) to work out how we could increase scalability in this area. Some ideas have been actioned and some are in the bag to pull out in future as and when needed.
We hope these updates are useful to current and indeed future customers. Please feel free to contact us to get involved with the continuing process of improving the agileBase platform!
For the start of April, we have a Spring refresh of the record editing interface for agileBase, making forms more friendly and breathing new life into the interface.
Continuing the previous work on form simplification such as ‘contracting blocks‘, this makes working in apps built on the agileBase platform an even more pleasant experience. There are various improvements to style and alignments, the most obvious being that form field outlines are changed from boxes to underlines, to reduce visual clutter.
In addition, administrators have a new UI tool in their armoury – coloured tabs. Tabs on screen can now be colour coded, to draw users attention or give them information. For example, if a complex process required half a dozen tabs, you may want to colour them green as each step is completed.
Setting this up works on views, in a similar way to many other features in agileBase. From the parent table of the tab, create a view which works out a colour for that tab, based on whatever criteria you like. The view should contain a text calculation which outputs a colour name, one of ‘blue, yellow, green, purple, pink, turquoise, coral, lime, orange, corn’. Alternatively any of the words for colouring calculations can be used.
Then, in the manage tab of the parent table, scroll down to the tab options and select the view you created in the ‘colour from view’ dropdown.
Since our last update on charting improvements, many customers have been trying out more ambitious charts on large amounts of data – great to see. We have had reports of a couple of limitations so have included some updates in this release to address those:
The Highcharts Boost module has been included to speed up chart rendering when there are lots of points
A serverside limit has been put on the maximum number of points to avoid overloads. To create massive charts we recommend using the API to integrate with a dedicated reporting tool such as Power BI or Zoho Analytics, which some customers use to great success.
Please continue to push the boundaries and we look forward to hearing what you do.
Earlier this month we released an update that allows the admin of a company to subscribe groups of people to charts, received by email daily, weekly or monthly.
Today’s update expands on that, adding these subscribed charts to every person’s visualisation dashboard. That means they can look at those charts in one place without waiting for a monthly (or other frequency) email.
Continuing the charting theme, another improvement is that charts now show totals for every series, which is particularly useful when charts are emailed or exported.
For users of our agile:NPDtech and agile:SA products, one of our projects on the horizon is to improve our customer success metrics, so these features will be being put to good use!
We hope these updates are useful to you immediately though. Please send any feedback to email@example.com – we look forward to hearing from you.
You may know that as a user of agileBase, you can subscribe to charts, to have them emailed to you monthly, weekly or daily.
For our product agile:NPDtech we’ve recently been working on customer success metrics so that customers can keep a track of how they’re doing after onboarding, where they’re seeing benefits and what they might want to focus on. As part of that work we wanted an easy way to send these metrics to groups of people, e.g. every user who’s a director or set up as a ‘project champion’.
So when editing charts, we also added the ability for an administrator to subscribe not just themselves but any role set up in the system. All users of that role will then receive that set of charts.
We’ve found this very useful ourselves, but the feature is now available in the platform for any customer to use! We hope that you also find a use and please let us know if you’ve any feedback once you’re tried it out.
Administrator set up
As an administrator, it’s easy to set this up. First make sure a role exists in the system with the users you want to target.
Then, in the tiles view, go to a view containing charts you want them subscribed to and click the visualisations button to see charts.
Click ‘send these charts monthly’ or whichever time period you prefer, then in the roles dropdown to the right, select the role you want to be subscribed. You can repeat this to add multiple roles. There are also other options you may wish to look at, such as the ability to include a set of data with each chart.
Finally, you can include some text to be sent in the report, if you want to include some background data on the charts or explain why they’re being sent. To do this, edit the view in the admin interface, go to the manage tab and write your text in the description box.
In other news, there’s been a user interface change that lots of customers have been requesting – the ability for the system to contract the view list at the top of the tile into a set of headings. Reducing large lists of views can make the system a lot more user friendly to new users, always one of our major aims, as well as reducing wasted screen space.
Moving the mouse over the view list will cause it to expand to show every view.
Please do contact us if you’ve any feedback or ideas for future changes.
With the Brexit deadline looming we felt the time was right to communicate what we are doing to ensure we remain compliant in case we crash out with no deal.
Perhaps the most obvious change we have needed to implement is driven by our desire to retain our EU based customers. Clearly leaving the EU doesn’t mean we no longer need to comply with EU regulations such as GDPR!
In practice, rather than just comply with the EU’s GDPR regulation themselves, we now also need to draft an additional contract per customer to say we are doing so!
Luckily the EU has created a range of “Model Clauses” for this very scenario. One these Models is specific to our situation as a SaaS provider and we can use this template, provided we make no changes.
If you are a food sector customer, to review this template please log in as normal at www.agilechilli.com and click agile:Space at the top right, to take you to the customer support portal. There, in the Documents tile, you’ll be able to download a copy, along with our GDPR and data security/privacy policies. Other customers, please contact us.
This is just one example of the changes we are expecting to need to make. We would be more than glad to hear from anyone who has any other Brexit concerns with our services and products.
A very quick update on a new feature for the agileBase platform: if you have a field which is a time duration, it can now be entered in hh:mm format. This can be useful for applications like time logging.
To set this up, create a decimal number field to use as the time duration. Edit the options and tick ‘use duration’.
In the user interface, people will see an hh:mm data entry box. However when data is saved, it will be stored in the database as a decimal number representing fractions of an hour. So for example
will be transformed into 1.5 i.e. one and a half hours. That means reporting, charting and data analytics can all be used to total up time.
We hope this is useful to you and look forward to hearing your comments.