Change and large systems

Generally, as systems get larger, it is more work to make large changes to them.

Building apps with an agile platform such as agileBase is very quick. However over the years, as new members of staff join, they may find it hard to work out how the system has been put together, which parts see a lot of use and which less.

Over the past months we’ve been working on addressing these things and we’re in the midst of designing some exciting further developments in that area too, but today I want to talk about the more technical aspects of making changes to very large and complex systems – what happens under the hood.

To understand why this is important let’s look at an example. Many businesses create calculations of component cost, VAT or other financial values, which then feed into many places of the system. Some customers have many hundreds of views, all referencing the same base calculation view.

What happens if we want to make a major change to that calculation view, such as adding or removing a field (gross and net figures as well as VAT perhaps)? 

What you may not know is that when you do this, the system automatically checks and rebuilds every dependent view. While this is happening, users and even the API are blocked from reading or writing data. Usually everything wraps up in a second or two and no one notices the time lag but sometimes, when there are hundreds of dependent views, it can take longer, causing a queue of waiting requests to build up and eventually time out, resulting in people seeing error messages.

Incremental view rebuilding

As of today, if the system works out that it’s going to be too slow to update everything in one shot, it will switch to updating dependent views one by one, in the background, allowing people to go on working.

Now, while a view is waiting to be updated, people will see a message like this:

The system will generally rattle through rebuilding all views necessary, so even when there are many hundreds, people should have to wait no more than a minute or so. And parts of the system which aren’t affected can carry on being used as normal.

Most importantly, these background updates will take a lot less resources while running.

We know that some administrators have previously found it necessary to wait until evening time to change views which many others are dependent on, to avoid the danger of overloading the system as lots of users tried to use a system at the same time as it was being modified.

Hopefully with this update, hard working system administrators can regain some of their work-life balance!

What else has changed?

A number of other improvements have taken place recently – here are some notable ones:

Comment fields usability improvements

The user interface for adding and viewing comments has been much improved, in particular comment boxes are now more differentiated from normal editable text boxes, making it obvious which is which.

Secondly, we’ve introduced ‘comment tags’. When a comment is imported from an email someone sends the system (e.g. to attach an email to a contact sales enquiry automatically by forwarding it or BCC’ing the system), it will get tagged with the sender/recipient of the email. Chaser responses will also get tagged and in future, we may use tags for other things, or let people add their own.

Dates in spreadsheet exports

When a view is exported to a spreadsheet, any date fields will now become proper dates in Excel, rather than text representations of dates, so you can quickly sort and filter them without having to do extra work to transform the cell type.

Record merging improvements

The record merging feature has seen a lot of use and has had some usability improvements and bug fixes. In particular, when a comment is produced due to one unique value overwriting another, that comment won’t be included in the comment stream on the homepage. These comments were polluting the comment stream when heavily used.

Inline editing improvements

Relation fields can now be edited when a view is in inline editing mode. Some customers are finding this indispensable for speeding up particular data entry tasks.

Major database performance improvements for agileBase

I always enjoy the chance to do some performance optimisation work on agileBase – especially when you figure out how to improve the performance and responsiveness of the system in a way that customers will definitely notice.

Now is a good time for us to be looking at this. Over the past year, more customers have started doing more complex things with greater amounts of data. Here’s how our database workload has increased over that time:

So here are a couple of updates that will be released next week:

1) Auto-updating cached views used for calculations

View caching is a feature first introduced in 2015. It can improve the responsiveness substantially – views which previously would have taken many seconds to search now return data in milliseconds.

A ‘primary’ view, that is one which a user looks at directly such as a sales report for example, is often a good candidate for caching. It can be set to refresh the cache whenever data changes so the user always sees up to date information.

However until now,  ‘underpinning’ views which a user doesn’t directly query, but which are used by those primary views (e.g. in joins) haven’t benefited as much from caching. Until this update, they wouldn’t automatically refresh when data changes, only updating once a day, meaning data that users see could be out of date – that’s not always an issue but sometimes can be.

An example from an actual customer is a complex and relatively slow VAT calculation (when run over many records), defined in a view ‘vat calculation’, which is then used in dozens of other views.

With this new feature, that underpinning VAT calculation view can be cached and set to update whenever data it depends on changes. Those multiple other views that reference VAT now have massively improved performance at a stroke!

This also works wherever a view is used in a chain. So if View 1 references View 2 which itself references View 3, View 3 can be cached and will update on demand whenever a user looks at View 1.

This one change should have a positive impact for many customers.

Once again, we couldn’t do this on our own, this was a real team effort. We have to thank two customers, Beacon Foods and Britannia Windows along with our partner Richard Wilson of Little House Consultancy for the time they spent in conversation with us as we progressed this idea.

2) Query Optimisation

Our second update is more niche. The feature will be enabled automatically, so administrators don’t need to configure anything. Technically it’s to do with re-writing the WHERE clause in the SQL that agileBase generates to take advantage of primary keys in certain situations – please contact us if you’d like to know the full details.

This change will only affect a few views, but one of those which will be improved significantly is currently taking about 20% of total query processing time over the whole system!

Our testing shows that each call to this view (and there are tens of thousands per day), which currently takes about 0.7 seconds, will reduce to a few microseconds, hence freeing a lot of server capacity for all users to benefit from.

Thanks must go to Oliver Humpage of Entuplet for being a part of the conversation which initiated this.

We always want to keep the system running like a well oiled machine for whatever scale of use customers need, so our work doesn’t stop there. Next on the list is looking at, amongst other things, scaling across multiple servers. We will keep you posted.

Retiring SMS-based two factor authentication

Keeping customers’ data secure has always been a primary concern here at agileBase. Following best practices such as automatically checking passwords against known leaks and enabling two factor authentication is a big part of that.

Since secure app-based two-factor authentication was released on the agileBase platform in 2018 we’ve been heavily promoting its use and gradually deprecating the older, text-message based authentication codes.

For example, for a number of months, app based authentication has been the only two factor authentication mechanism available for new users.

Information has come to light in the last couple of days that the older SMS based authentication mechanism is fatally flawed from a security perspective. Although it’s been known for a while that SMS codes have certain issues, using them has been seen as better than nothing. Exploiting the issues has required a certain amount of work and expertise. However recently, there have been cases where a newly discovered exploitation technique allows customers’ authentication codes to be easily stolen without them even noticing anything’s amiss.

Given this new vulnerability example, we feel that the time is right to retire SMS codes entirely, as they increasingly provide more of a sense of false security than actual security.

As of the next release, agileBase will stop sending authentication codes by text message. Anyone using this mechanism will revert to having no 2FA, so they will be prompted to set up app-based two factor authentication after logging in.

Due to past initiatives this change will only affect the 12 percent of people using the old SMS mechanism.

User interface makeover

From Monday, when you log in to the system, you may notice some changes in the user interface. It’s had a bit of a makeover to make it easier to use, more modern and make navigating  around much quicker.

Here’s a quick rundown of the main updates, with a couple of annotated screenshots.

Here’s a tile which has been opened – ‘Ingredients’.

Firstly, there’s a toolbar at the top of the screen, which remains visible wherever you are in the system. It will stay there as you scroll up and down through data, so will always be easily accessible.

At the top left, are three navigation options

  • The agileChilli logo will always take you to the homepage, wherever you are
  • The back arrow to the right of it, will take you back one step to the previous screen you were in (finally taking you back to the homepage)
  • The history icon (the clock) next to that is a quick way of navigating anywhere else in the system without needing to go back to the homepage. People testing the system have loved this, it’s shown in more detail below

The menus to the right are specific to the tile that’s open. The system can cope with any number. On a typical laptop, there’s space for eight to ten (larger screens may see more) to be seen initially. If there are more than that, the toolbar will automatically expand to show them when you hover over the area.

Near the top right is an actions panel, with actions relevant to the view or record on screen. In a view, the buttons allow you to

  • show and add charts
  • export to spreadsheet, or share a link to the view with other users (this is a new feature)
  • add a new record

The filtering button shows or hides the filter boxes. The current state will stay when you switch between tiles, so you won’t have to keep clicking it if you do a lot of filtering. Hiding the filter boxes is also a quick way of clearing all filters applied to the view.

In general, there’s more space given over to content – so you can see more rows and columns on screen at once.

Finally, Windows users will find a welcome change – you can now scroll quickly right and left using scrollbars, rather than using the old buttons.

History

Here’s a look at the history feature mentioned earlier.

When you click the history icon, a large, searchable menu appears.

At the top is a list of the recently opened records, which you can navigate to quickly.

Below that are the recently opened views. Again, you can click on them to jump directly back to them.

The search box lets you look at not just recently opened views and records, but any tile or view in the system. For example, if you want to look at an allergens report for your ingredients, you can now type ‘allergens’ into the search, rather than navigating to the homepage, opening the tile which contains the view (which you may not be sure of) and then selecting the right view.

That’s a quick run-down – everything should be reasonably intuitive but it is different to before, so we thought you’d appreciate this heads up.

There are plenty more improvements coming down the line following this visual refresh, we have some exciting things in the roadmap which should really shake things up, particularly making building apps easier. We look forward to launching those.

In the meantime, please let us know what you think when you use your system the next time. Feedback is really useful and we will always try to react quickly to any good suggestions.

Charting and reporting improvements

Continuing on full steam ahead with agileBase platform development, we have another couple of features to announce this month, to make the system more useful when generating and distributing reports.

Weekly charting

Firstly, when creating a chart, you have the facility to total (or perform other maths such as averaging) data by year, quarter, month or day. We’re now adding a fifth option, by week of the year.

That’s particularly useful if you configure the system to send out weekly reports to people. The charts can then show the past week’s figures, compared to the week before, or the same week last year.

Subscribing to spreadsheets

We all know that spreadsheets are the lingua franca of data. People with the necessary privileges can export to spreadsheet at any time in order to send the data to others who may not have an agileBase login, or to import into other systems like accounts packages for example.

We’ve now made it easier to automate this process. Just as you can subscribe yourself, or a group of people, to receive charts daily, weekly or monthly, you can now include a spreadsheet of raw data with those charts. Recipients will receive two attachments, one PDF containing the charts subscribed to and a second Excel spreadsheet containing the data.

This takes effect automatically when you opt to include data with your charts, if more than the top 20 rows are requested. The data included can optionally be filtered by date, to show just those records added or updated since the last time the data was sent.

Although sharing spreadsheets will work for many, sometimes a more automated solution is beneficial. For example in cases where data is destined to be imported from one system to another, removing the spreadsheet in the middle can reduce manual work and drastically speed things up.

In those situations, we do encourage customers to make use of the comprehensive API capabilities, which will allow you to fully automate integrations with third party systems, such as websites, enterprise-wide reporting, or accounting packages.

Merging duplicate records in a CRM system

In the latter part of 2020 we published a post explaining one way of finding duplicate company names (or any duplicate information) in your database.

Once they’re found, of course, the next step is likely to de-duplicate the information, combining the data from two records, leaving only one.

Doing this manually can be a time consuming process if the data structure is complex. For example, when merging two organisation records into one, you may have to not only check all the fields (company name, website, various categorizations etc.) but also ensure that all related ‘child’ data such as contacts, sales opportunities, addresses etc. are attached to only one record.

For example, say we have two companies in a suppliers database, one called ‘ABC Goods’ and another called ‘A.B.C. Goods’. ABC has two contacts, Alice and Bethany, whilst A.B.C. Goods has one contact, Carl. We would want to end up in a state where there was only one company, say ABC Goods, with all three contacts linked to it.

This month, we’re launching a facility which does all the work automatically for you.

Using it is a simple process:

  1. load up one of the records that you want to merge with another – we will call this record 1.
  2. Click the clone/merge icon at the top of the screen and select ‘merge with another record’.
  3. Click the record you wish to merge in (you may need to filter to find for it). We will call this record 2.

agileBase will then merge data from 2 into 1, then delete 2. Wherever 2 contains data in a field which is blank in 1, 2’s data will be copied in. If 1 and 2 both contain different data for a field, a prompt will be shown asking you to choose which one to keep. Note for tags fields, it’s also possible to keep both.

Any ‘children’ of 2, e.g. contacts of an organisation, will be moved to 1.

At the end, you will be left with only record 1. Record 2 contains no data that hasn’t been copied to 1 (or that you’ve chosen not to copy), so will be deleted.

Note you can cancel the process at any point.

It’s possible of course that 1 now contains duplicate child data. For example, if ABC Goods actually had three contacts, Alice, Bethany and Carl, whereas A.B.C. Goods had just Carl, then after merging, you’d have one company ABC Goods containing, Alice, Bethany, Carl and Carl, i.e. two versions of Carl.

To sort that out, just repeat the process for the two Carls, i.e. use the record merge facility to combine them.

Administrator control

As an administrator, you can enable or disable the merge facility on a per-table basis.

Edit the table options and tick or un-tick ‘allow record clone and merge’. The clone and merge facilities are grouped together currently. If you wish to separate them for your users, please let us know.

Other improvements in this release

It’s been just over a month since the last release and as well as the new record merging facility, there are a handful of other new features and bug fixes, notably:

  • Any hyperlinks (e.g. links to Zoom or Teams meetings) in a calendar event will now appear in the calendar ‘quick edit’ popup that appears when you click on an event
  • A bug to do with importing emails has been fixed. Forwarding or sending an email to the system should now correctly attach that email to the relevant CRM contact in all cases

agileBase release – user experience improvements

Today there’s one major change to make you aware of and in total 68 issues and ideas have been acted on since the last release. We’ll just pick out the highlights here.

Tab lists

This affects tabs which have multiple rows under them (e.g. in an organisation record, the ‘contacts’ tab may have many contacts).

In that case, when you click on a row (e.g. a contact), that record (the contact details) will now replace the list of contacts. New navigation buttons allow you to quickly return to the list, or navigate to and fro between each contact. Here’s a screenshot of what it looks like:

This simplification makes the screen a lot more easy to consume and understand. If there are any questions please get in touch.

Other user interface updates

No more field level comments

NB it’s no longer possible to add comments next to any field. This was a little-used feature which was causing issues in cleaning up screen layouts.

Of course, comments can still be added to any record with a dedicated comments field. If there is no comments field in a particular table, you may need to add one to allow comments to be added.

This doesn’t affect comments added when unlocking a record, which will still work.

Auto-generated sequences in titles

If a table contains an auto-generated sequence number field, that number will be used as part of the record title. In fact it will be moved from the form up to the title.

So an invoice with an auto-generated number might be titled ‘Invoice 12345’ for example.

Rich text editing

Bullets and numbered lists can now be used when entering text into large boxes.

Page component improvements

On-screen controls like buttons and the ‘record locked’ notification have had a makeover to make their intention clearer and so that they fit in better with the rest of the system.

It’s now also possible for workflow buttons to be set to a ‘disabled’ state. You may want to disable a particular button until certain fields have been completed for example.

To control a button’s disabled/enabled state, use the ‘lock when record in view’ option for the referenced field representing the button. In this case the ‘locked’ state represents a disabled button, unlocked is enabled.

The print/export button has now been changed from a ‘print’ icon to an ‘export’ one, as exporting to spreadsheet is a more common action than printing

Password reset process

The clarity of graphics and wording throughout the password reset process has been improved

Security improvements

Notably, system administrators will now be emailed whenever a user is granted access to a table which is marked as containing commercially sensitive or personal data.

Bugfixes

Notable fixes:

  • PDF previews now work correctly in a document library tile
  • Date pickers appear properly in the Firefox browser
  • Chart subscriptions i.e. the regular sending of charts by email sometimes failed – that feature now works reliably

Miscellaneous

  • The API now accepts dates including milliseconds, e.g. “2020-08-20 08:14:52.014”

An agileBase example: de-duplicating company records

This week we have been asked by a customer to do a bit of work that’s interesting enough to write a blog post about – always nice!

Preventing or removing duplicates is a common requirement in CRM systems and other database apps. With a multi-user system, it’s not uncommon for two different people to try to add the same company details twice for example, without checking whether they’re already in the database.

On the prevention side, there are a couple of features of agileBase which help. The first and most obvious is the unique option for fields. With this enabled on a company name field for example, people will be prevented from entering the exact same name twice. If appropriate this can be slightly improved by also turning on the text case option to force all entries to the same case as they’re entered.

However, even this isn’t going to catch all duplicates. Company names can often be entered slightly differently, e.g. with ‘Ltd’ or ‘Limited’ following the name, the word ‘The’ at the start or with different punctuation.

To catch these, the build in duplicate detection can be used, activated for all record title fields. When someone enters a name, the system automatically searches all existing records for any similar names.

Reporting existing duplicates

Even with all these features we will often still want to find duplicates that in one way or another have made their way into the system. Maybe the data hasn’t been manually entered by staff, but imported, e.g. from spreadsheets, or via an API from other software or a website. However, in our case, the cause was the merger of two different business units and their data.

Finding ‘close match’ duplicates is an interesting problem that can be solved in a number of ways. Here’s a short introduction and there are many articles and papers which go into the mathematical details of various methods.

We chose to use trigram matching, an approach which is supported natively by the database agileBase uses, PostgreSQL and which is fast and effective. As they say, “this simple idea turns out to be very effective for measuring the similarity of words in many natural languages”.

A trigram is a set of three consecutive characters, for example ‘agileBase’ would contain ‘agi’, ‘gil, ‘ile’, ‘eBa’, ‘Bas’ and ‘ase’. There are some further considerations, for example how to treat the starts and ends of words, which are explained well in the PostgreSQL documentation page.

In brief, we simply count the number of trigrams two words or phrases share as a proportion of the total number.

Our goal

So as a concept, what we want to do is compare every company name in our database with every other company name, get a similarity score from the pair and sort the results in order of descending similarity, with a cutoff to stop showing results once they fall below a certain similarity threshold.

An immediate thought is ‘how many comparisons is that’? Well, the database we’re looking at currently has 13,000 companies in it. That would mean 13,000 ^ 2 = 169 million comparisons. Ideally, we’d like the system to continue working well as the number of records significantly grows. At 100,000 records, the number of comparisons would be ten billion.

As this ‘naive’ number of comparisons is the square of the number of records, we’d like to get that down as much as possible in order for performance not to suffer.

We can immediately cut the number in half – if we’re comparing Company A to Company B, we don’t then also need to compare Company B to Company A.

We can also use the fact that duplicate names are very likely to start with the same letter. There are some exceptions, for example Bristol University and University of Bristol (only the latter is correct!) however that will be uncommon and the tradeoff is likely to be worth it. To take advantage of this fact we will also need to strip any common prefixes from the names to compare, for example ‘The’ as in ‘The University of Bristol’.

Assuming an even distribution of first letters, the number of comparisons would be 

(((n/26)^2) x 26) / 2.

(also taking into account cutting the number of comparisons in half as above).

For 13,000 names, that would result in 3.25 million comparisons, down from 169 million. For 100,000 it would be 192 million, down from 10 billion. Still quite a lot, but we can give that a go and see how it works out.

Actually, thinking about it, if we import more sets of data from other sources in future, we will only need to compare the newly imported companies with all others, not do a full comparison of everything with everything. So I don’t think we’ll ever get anywhere near the top end of those numbers.

There are other reduction methods you’ll be able to think of – for example, comparing only those where postcodes are the same (assuming each company has only one address, or HQ details are all filled in). However the above is what we went with.

Implementation

Ok. Let’s move on to making this work.

Our first step is to create two views, which we can then compare with each other. Each view will have the ‘cleaned’ version of a company name and the first letter of that name. Let’s add two calculations

simple name = 
regexp_replace(
    lower({a) companies.company name}),
    ‘^the\\s’,
    ”
)

first letter =
substring(
    {simple name}
    from
      1 for 1
)

These calculations use SQL to 

  1. convert the name to lowercase and strip any leading ‘the’
  2. extract the first letter of the resulting ‘simple name’

Once we have these two views, which we’ll call ‘similarity check 1’ and similarity check 2’  we can join them together and do the similarity calculation.

From similarity check 2, we add a complex join on the ‘first letter’ calculation to the first letter calculation from similarity check 1.

Then also in similarity check 2, we can add a boolean (true or false) calculation to allow us to cut the number of comparisons in half:

first comparison = {similarity check1.id:companies} < {companies.id:companies}

(here, the name of the table containing the company names is, well, ‘companies’). This calculation outputs true if the company on the left hand side has a lower internal ID than the one on the right, or false if not.

We now just add a filter on that calculation:

first comparison = true

Now to the similarity check itself, which is simply

similarity =
similarity(
    {similarity check 1.simple name},
    {similarity check 2.simple name 2}
)

All of this has been in the ‘similarity check 2’ view. We now have all the information we want to look at in one place. Well, almost – for people to actually use the system, they’ll probably want to look at the actual company names, not our simplified versions. They may also want to see other information, like perhaps the account manager for each company.

So we’ll create a third view, just called ‘company duplicates’, which pulls in the calculation from ‘similarity check 2’ but displays things in a more user friendly manner.

‘company duplicates’ joins to similarity check 2 on the internal company ID. We then sort and filter on similarity, using a filter ‘similarity > 0.7’ in our system.

I’ll leave the details of adding additional fields such as account manager etc. It is worth noting that to speed things up for users, we set ‘similarity check 2’ to cache its results. On our dataset it takes a few seconds to load – with the results cached, users can view and search the data with better responsiveness. The load on the system is also reduced.

Here’s what the results look like (details have been changed or pixellated, but you can see the first couple of example names compared.

Hopefully this can act as a useful pattern for people to follow if they want to do similar de-duplication work.

The next step could be to tackle the process of actually merging and deleting records once duplicates have been identified. For example, workflows could allow details of two company records to be automatically merged together. However that is a task for another day!

Calendar and other updates

It’s been slightly longer than normal since our last release of the agileBase platform, but to balance that there’s tons of stuff packed into this one.

We’ve been working closely with partners to make this a significant update with lots of usability improvements. There aren’t that many new features and users shouldn’t be impacted by any of the changes, but many should find the system easier to use and more capable.

Improvements for users

Calendar updates

First up is the built in calendar. 

  1. Drag and drop is now supported for events, so it’s much easier to reschedule things. If an event is dragged from one day to another, the time of day remains the same. Similarly, if an event is moved from one month to another, the day of the month remains the same. Events can of course be edited by clicking on them to fine-tune any details.
  2. Event editing popup: clicking an event will pop up a small window to edit the main details of an event. You can then click through to the full record if you like, but making simple changes to e.g. an event title or time is a lot quicker.
  3. Weekends are now always shown, but smaller. Previously you could toggle them on and off by clicking the ‘month’ selector at the top of the screen, but this wasn’t particularly obvious to new users. Events can now easily be scheduled on weekends if necessary, but the space assigned is half the normal day width to allow more content to be visible in week days.
  4. A new year planner view is now available. This shows an entire year on one screen in a ‘timeline’ format, letting you schedule projects, milestones or other events easily. It works really well with the new drag and drop capability too.
  5. Calendar drill-down: individual day numbers and week numbers are now clickable, navigating you to a day or week view as appropriate, for the period selected.

Simple form layouts

There’s a new layout option for simple forms, which hides the first tab if it has no contents. This is easier to show with an example. Here’s what a simple form may have looked like before. It has just a handful of fields, no blocks and a tab.

Here’s what it looks like following the update. The unnecessary first tab is removed and the remaining tab is selected by default (if there is more than one tab, the first one will be selected by default).

To hide the tab, just ensure that there is only one separator field in the table and that it’s the last field (or at least that there are no visible fields after it).

Customising chart subscription times

Users can select to receive certain charts by email – https://docs.agilebase.co.uk/docs/charting/#subscribing-users-or-roles-to-charts

It’s now possible for people to select what time of day they’d like their charts to arrive, which may be useful for e.g. shift workers who need to see particular sets of data when they start work.

A user can edit their profile from the homepage to do this, or an administrator can set the time for them by editing their user details

Improvements for administrators

Simpler multi-stage workflow setup

Often, workflows are made up of a ‘chain’ of individual actions, For example, if a new enquiry comes in from a website form (via an API), you may want to 

  1. create a new organisation
  2. add a contact to that organisation
  3. add a sales lead, related to that contact and organisation

Previously, to link these three records together, you would have to add hidden fields to record the fact they all came from the same web enquiry along with lookup calculations to link them to each other.

Now, you can just tick ‘auto-set relations’ in the workflow settings of steps b and c. When a workflow creates a new record, it remembers which one it was. Later workflow steps in a chain will automatically link to it, where a relation exists.

Renaming fields

Previously, if a field was used in an email notification, chaser or ‘create record’ workflow, renaming that field could break parts of those things, requiring you to manually find and fix each.

Now, the system will automatically update references to the field in the background, whenever a field is renamed.

Assigning users

Currently, using the ‘list of users’ option (https://docs.agilebase.co.uk/docs/fields/field-options/text-field-options/#lists-of-users) you can assign a user to a record e.g. as an account manager or responsible person. 

Now, you can narrow down the list of users to only those with a particular role e.g. managers. This can work with multiple roles too – please see the documentation link for more details.

Two Factor Authentication (2FA)

Some users have been finding text-message based 2FA problematic, in that text messages take a while to arrive, or never arrive at all. This can be for many reasons, for example lack of reception, problems with the phone network or the fact that a valid mobile phone number hasn’t been entered (e.g. if a landline is entered, that won’t work).

For those reasons, 2FA by text message has been deprecated. It will continue to work for users who already have it set up, but new people, or those who don’t have a phone number entered, will only be able to use app-based 2FA, which is far more reliable.

We will continue to monitor 2FA usage and in the longer run are keeping a close eye on the progress of upcoming technologies such as webauthn, which allow biometric or hardware key authentication mechanisms.

User interface updates from partners

It’s always good to have new eyes on the platform, and this month we’ve been working on projects with a couple of new partners – or at least supporting them while they do most of the work! 

During this period they have noticed a couple of improvement opportunities in the user interface, so this release is a minor one to implement those. Thanks to Argenta and Techni-K for their input.

Read-only mode

Certain people e.g. contractors or external parties may only need read-only access to some data in the system. The read-only version of the record screen has now been improved to remove some glitches, such as duplication of the field names of checkboxes and make everything look better.

Embedded data fields

An option has also been added for referenced data fields to allow you more control over how they’re displayed on screen. 

Below is how each of the four options will look like on screen to a user.

Panel fieldname and content fieldnames:
Panel fieldname, no content fieldnames
Content fieldnames
no fieldnames

To use this new functionality, go to the fields tab for the table the field is in and click ‘edit options’. Then choose a value for the ‘display of fields’ option.

Optional colouring for date fields

The system automatically highlights out of the ordinary values with colour, based on how many standard deviations from the mean they are. This works with both numbers and date fields.

For numeric fields, you have always been able to enable/disable this behaviour in the field options. That has now been extended to date fields too. Previously it was always enabled, now it can be disabled if necessary.

More intuitive navigation of sub-records

Navigation arrows have been added to the headings of sub-records (records in a tab of the main record – ‘black pepper’ in the below example), to make looking through many records quicker and more obvious.

We hope these small tweaks make a difference. We look forward to hearing more of your feedback in future.