Performance of Google Apps Script Text Finder Class on 2 Approaches to Searching Large Datasets

Inspired by research into a recent blog post, the Google Apps Script Text Finder Class’ Find All (findAll()) and Find Next (findNext()) methods were benchmarked over two different datasets containing 50,000 rows. The first dataset contained 1,000 cells matching the search text. The second dataset contained 100 matching cells.

For each dataset, a test was conducted to retrieve either the first 10 matching cells or the first 100 matching cells. The Find All and Find Next approaches were tested and compared on each test.

It was expected that Find Next would perform best on the condition where the dataset contained a large number of found items and only a small number of first cells needed to be reported. The benchmark results suggest that this hypothesis is most likely.

First number of cells to retrieve Test Function Avg. run time over 100 runs. Fastest Function Fastest Avg. Time Avg. time Difference
1000 items to find
10
1
v2 findAll 1626.24 v3 findNext 1368.45 257.79
v3 findNext 1368.45
50
2
v2 findAll 1578.19 v2 findAll 1578.19 4993.61
v3 findNext 6571.8
100 items to find
10
3
v2 findAll 360.94 v2 findAll 360.94 975.16
v3 findNext 1336.1
50
4
v2 findAll 377.13 v2 findAll 377.13 6175.59
v3 findNext 6552.72

Table: The average time in milliseconds of 100 runs of each test of Apps Script Text Finder findAll() and findNext() methods. Image link for mobile-friendly viewers.

Method

Sample Data

Two columns of data 50,000 rows deep were generated for this test. Each cell in each column consisted of a number; either 1, 2, 3, 4 or 5. An equal spread of numbers 1 through 4 where added to each row. Each column differs by the number of 5s in each row:

  • Col A: 1,000 5’s
  • Col B: 100 5’s

Each column was then selected and randomised with: Data > Randomise range.

Test

Two functions are compared to test their performance based on four test conditions based on 100 runs of each test:

  1. Retrieve the first 10 cells containing the search text where the range contains 1,000 matching search items.
  2. Retrieve the first 50 cells containing the search text where the range contains 1,000 matching search items.
  3. Retrieve the first 10 cells containing the search text where the range contains 100 matching search items.
  4. Retrieve the first 50 cells containing the search text where the range contains 100 matching search items.

The time in milliseconds was recorded using the JavaScript Date.now() method before and after the functions were run. The difference in time in milliseconds was then appended to an array and added to a Google Sheet column for each test type. This culminated in 8 sets of 100 results.

The average of each test was then recorded and used to compare performance.

Note: Performance.now() is not available in Google Apps Script. 

Code

All code and results can be found copied from this sheet:

Analysis of Google Apps Script Create Finder Class Retrieve n found values

To explore the code and run your own independent tests, make a copy of the Google Sheet: File > Make a copy.

More detailed breakdowns of the code for each test function can be found in the source tutorial.

Note! There is no v1. The version numbers refer to the tutorial related to this post.

Main Test RUN

This function ran all the test conditions. Modify colPastePosition to add the culminated times to the desired columns. Then uncomment the desired run.

test_v2 – Google Apps Script Text Finder Class- findAll()

Code breakdown can be found here: link.

This function retrieves the full list of all found cells using the findAll() method from the Text Finder Class. All available found items in the range are then stored in the found variable.

It then relies on a for-loop to iterate through each cell and collect the cell location using the Spreadsheet App Class’ range getA1Notation method. Each cell location is then stored in the locations variable as an array item before returning the array to the initialising function.

The for-loop breaks when the total number of required cell items (the position) equal the index variable (i) in the loop.

test_v3 – Google Apps Script Text Finder Class- findNext()

Code breakdown can be found here: link.

In this function, a call is made to the spreadsheet to retrieve the found cell value each time findNext() method of the Text Finder Class is called. On each iteration, the getA1Notation method is used to retrieve the cell location. This location is then stored as an array value in the locations variable before being returned to the initiating function.

The function used a while-loop to iterate through each next item found until the counter – or the number of required cells to collect – is reached.

Results & Discussion

Analysis of Google Apps Script Create Finder Class Retrieve n found values
Performance in Milliseconds to Retrieve the first 10 or 50 Matching Values over a 50,000 Row Range Contain Either 1000 or 100 Matchable items Using the Google Apps Script Spreadsheet App Finder Class.

Test 1: Retrieve the first 10 cells containing the search text where the range contains 1,000 matching search items.

Version 3 –findNext() performed better on average when there were 1000 potential items to find in the range but only the first 10 items need to be selected. Versions 3’s average speed was 1368.45ms compared to version 2’s average run speed of 1826.24ms. This is a performance increase of 257.79ms for version 3.

Version 2’s lower performance is likely due to needing to collect all available found cells before it can extract the top 10 items.

Version 3, makes 10 calls to the Google Sheets in this example. Compared to version 2, this takes relatively less time than collecting all available found cell references to the search item.

TEST 1: 1,000 randomised items to find in 50,000. Return first 10 matches using v2-findAll and v3-findNext functions.
TEST 1: 1,000 randomised items to find in 50,000. Return first 10 matches using v2-findAll and v3-findNext functions.

Test 2: Retrieve the first 50 cells containing the search text where the range contains 1,000 matching search items.

Version 2 – findAll() performed significantly better over 100 runs than version 3 when retrieving the top 50 found cells from a possible 1000. Version 2 was, on average, 4993.61ms faster at an average runtime of 1578.19ms compared to version 3’s sluggish 6571.80ms average.

It was expected that test one and test two’s times for version 2 would be similar and there are only 48.05ms between their average runtimes.

Version 3’s poor performance is likely due to its reliance on calling the spreadsheet to collect the cell data on all 50 calls it needs to make.

TEST 2: 1,000 randomised items to find in 50,000. Return first 50 matches using v2-findAll and v3-findNext functions.
TEST 2: 1,000 randomised items to find in 50,000. Return first 50 matches using v2-findAll and v3-findNext functions.

Test 3: Retrieve the first 10 cells containing the search text where the range contains 100 matching search items.

Version 2, again, performed better by 975.16ms than version 3 when there was a smaller potential number of items to find in the range and only the first ten items need to be retrieved.

Here the performance margin between the two versions was closer than in the previous test. Version 2’s average run speed was 360.94ms while version 3’s runtime was 1336.10ms.

With a smaller number of retrieved items, the version 2 findAll() function did not have to work as hard to collect the methods related to each range it collects. Whereas version 3 still needed to make 10 performance-intensive calls back to the Google Sheet each time with relatively no performance change to test one.

TEST 3: 100 randomised items to find in 50,000. Return first 10 matches using v2-findAll and v3-findNext functions.
TEST 3: 100 randomised items to find in 50,000. Return first 10 matches using v2-findAll and v3-findNext functions.

Test 4: Retrieve the first 50 cells containing the search text where the range contains 100 matching search items.

Predictably, version 2 – findAll() performed the best when the expected match sample is small (100 available matches) and the total first set of cells to retrieve was relatively large (50).

Version 2’s average completion time was 377.13ms compared to version 3’s average of 6552.72ms, performing on average 6175.59ms faster. This is by far the largest margin on performance between the two versions.

Here again, version 3 must perform 50 calls to the Google Sheet, each one retrieving the cell range data. Alternatively, version 2 makes one call to the spreadsheet and then retrieves the cell data for all collected values. This is significantly faster than version 3’s approach.

TEST 4: 100 randomised items to find in 50,000. Return first 50 matches using v2-findAll and v3-findNext functions.
TEST 4: 100 randomised items to find in 50,000. Return first 50 matches using v2-findAll and v3-findNext functions.

Overall

On datasets that may have the potential to contain a large number of matching items, but fewer required results to return, version 3 may be the best option. In all other cases, version 2 is the most optimal approach to finding data in a range.

It is important to note that it can be difficult to accurately measure performance with Apps Script runs because resource allocation to run a script does seem to vary. Nevertheless, with a sample size of 100 runs, it is hoped that average values will be more accurate than a smaller sample.

Grab Your Own Copy of the Google Sheet and Attached Code here

If you have found the tutorial helpful, why not shout me a coffee ☕? I'd really appreciate it.

Create and Publish a Google Workspace Add-on with Apps Script Course

 

Looking to learn more about Google Apps Scripts in a more structured format? Udemy has some great courses that can get you from the basics to a real Google Apps Script pro.

Got a more specific problem you need help with, but don’t have the time to develop the skills? Make an enquiry on my 'Hire me!' page. I occasionally pick up projects. Alternatively, Fiverr’s your best bet to find a skilled Google Apps Script developer to solve your problem quickly and professionally. *

*The above affiliate links have been carefully researched to get you to what you specifically need. If you decide to click on one of these links it will cost you just the same as going to the site. If you decide to sign up, I just get a little pocket money to help pay for the costs of running this website.

~Yagi

 

Find All Values in Google Sheets with Apps Script

Just like when using the Ctrl + F shortcut in Google Sheets to find values in your spreadsheet, there is a class in Google Apps Script that can do the same thing.

This could be a useful tool as a part of an automation process. For example, finding the location of a value and applying formatting to it or copying the cell’s entire data into a separate location if the value is a part of a larger text in the cell.

This tutorial accompanies the YouTube video tutorial of the same name.

Grab a copy of the starter sheet to play along and get the most out of the video.

Starter Sheet

Find All Apps Script – Starter Sheet

If you have found the tutorial helpful, why not shout me a coffee ☕? I'd really appreciate it.

The Code

Note that for the video tutorials, I have added the variables (e.g. the Sheet Name, Range) inside each function. It is usually good practice to keep these functions independent and call them from other functions. This makes them more reusable.

Instead, we can add the variables we need as parameters for the functions and return the result. See the example below:

These examples use the TextFinder class as a part of the Google Apps Script Sheets App Class.

In these examples, we use the findAll method of this class. This will return an array containing all the cells containing the selected value searched. From here, you can treat each cell as a range and call range methods like:

  • Get A1 Notation.
  • Get Sheet – Get Name.
  • Get Row.

We use the JavaScript Map method to iterate through each item that we find.

Find All Values in All Sheets

This finds all values in all sheets and returns an array containing an object for each sheet containing the sheet name and the cell location.

Returns:

 

Find All Values in Selected Sheets

Finds all the values in a selected sheet and returns an array identifying the cell that each item is found.

Returns:

[ 'A3', 'A4', 'F14', 'A16' ]

Find All Values in a Selected Range

Finds the values for any item in a selected range and returns the row the item was found on.

Example 1

Returns:

[ 3, 4, 16 ]

Example  2

Returns:

[ 1, 2, 4, 5 ]

The Video

Create and Publish a Google Workspace Add-on with Apps Script Course

 

Looking to learn more about Google Apps Scripts in a more structured format? Udemy has some great courses that can get you from the basics to a real Google Apps Script pro.

Got a more specific problem you need help with, but don’t have the time to develop the skills? Make an enquiry on my 'Hire me!' page. I occasionally pick up projects. Alternatively, Fiverr’s your best bet to find a skilled Google Apps Script developer to solve your problem quickly and professionally. *

*The above affiliate links have been carefully researched to get you to what you specifically need. If you decide to click on one of these links it will cost you just the same as going to the site. If you decide to sign up, I just get a little pocket money to help pay for the costs of running this website.

How to Share a User on a Published Google Site with Google Apps Script

There are times when you might just want to share a user on a published Google Site as a part of a Google Apps Script Automation. Say you have an internal Google Site for a project and you want to share it with a member as a part of the onboarding process. Alternatively, you might want to share your site as soon as you receive a Stripe payment webhook ping.

To share a user in Google Sites you need to select the share icon from the top menu of the site editor. Then in the dialogue box add the user’s email and choose Published Viewer.

Give Published Viewer access to a Google Site

While there is no specific Google Apps Script API to manage Google Sites, we can share permissions and accessibility with Google Drive.

Continue reading “How to Share a User on a Published Google Site with Google Apps Script”

Count the Occurrence of a Selection of a Cell in Google Sheets with Apps Script

A mate reached out to me last week asking if there was a way to monitor the times any article in a Google Sheet list goes through an editing stage before being published. From there, he wanted to report the number of edits on the article in an adjacent cell.

Take a look at the example:

As you can see in the example, whenever “NEEDS EDIT” is selected from the dropdown menu, then the counter column for the selected row item is updated automatically.

In this tutorial, we will cover how to create this for a single Google Sheet tab or for multiple selected tabs. I’ll also wrap up with a script to calculate just the aggregate count of changes to all the rows in the selected range and report it in a single cell.

Check out the video tutorial that covers the basics and grab the Starter Sheet to play along:

The Video

Scheduled release 6 Dec 2022 9am GMT+10

Generally better to view it from YouTube.

Starter Sheet

Here is a link to the starter sheet copy the sheet and head to Extensions > Apps Script in the menu.

Starter Sheet

The onEdit(e) Simple Trigger

Everything starts with the onEdit(e) simple trigger. This is a special trigger in Google Apps Script called a Simple Trigger that will run every time you edit your Google Sheet. If Apps Script sees the onEdit() function in your project it runs automatically.

Every time onEdit() is executed, it will retrieve an event parameter that is commonly referenced as "e" (onEdit(e)). The event object can return things like:

  • The current value
  • The previous value
  • The edited range
  • The source spreadsheet
  • The trigger id
  • The authorisation mode

We will be using a few of these in our code below.

Unless you have a very short coding task, it is generally good practice to reference another function that will be executed when onEdit() is run rather than coding directing within the onEdit() function.

Take a look at the three examples I added for my test project for this tutorial:

You can see that each one of these functions will be run one after the other when a cell is edited and calls onEdit().

Count Every Time The Cell Changes

In this example, we add a count to our adjacent counter cell each time the cell is edited.

Note that we take the event (e) as our parameter for our countEveryTimeCellIsChanged(e) function.

Set the Variables

First, let’s assign all our variables at the top of our function to make it easier for us to change should we need to down the track.

We have 3 variables that we need to include here. They won’t change so we will make them a constant variable:

  1. tgtSheet – This is the target sheet tab name. For our example, this is assigned to “Single List”.
  2. tgtCol – This is the target column or the column that has the values that will need to be monitored for changes. In our example, this is the status column, column 3.
  3. counterCol – This is the column where we display our change count. For us, this is column 4, but it could be any selected column in the sheet.
Numbers correspond to the points above.

You can stop here if you want to copy and paste the script into your own project and update the variables for your own purposes. Or continue to learn how it all works.

Create and Publish a Google Workspace Add-on with Apps Script Course

 

Looking to learn more about Google Apps Scripts in a more structured format? Udemy has some great courses that can get you from the basics to a real Google Apps Script pro.

Got a more specific problem you need help with, but don’t have the time to develop the skills? Make an enquiry on my 'Hire me!' page. I occasionally pick up projects. Alternatively, Fiverr’s your best bet to find a skilled Google Apps Script developer to solve your problem quickly and professionally. *

*The above affiliate links have been carefully researched to get you to what you specifically need. If you decide to click on one of these links it will cost you just the same as going to the site. If you decide to sign up, I just get a little pocket money to help pay for the costs of running this website.

Get the Correct Sheet Tab and Column

Lines 33-38

We don’t want to end up counting the wrong column or updating a completely different sheet tab by mistake. This means we need a way to check if we are in the right spot.

The first thing we need to do is check the range we are currently editing. This can be done by grabbing the range object in our event parameter. Line 33

We now have access to the Google Apps Script range class. From this class, we can get the sheet class and also the range column numberLine 34-35

Using the JavaScript ‘if’ statement we first check that the sheets name (sheet.getName()) is equal to our assigned target sheet name and if it is, then check that the selected column is equal to the target column. Line 38

Update the Counter Cell

Lines 40 – 48

We need to update the adjacent counter cell in the same row that was changed when edited.

We can use the range class again with the getRow() method. This will give us a row number. Line 40

const row = range.getRow();

Next, using the sheet variable we created earlier we can call the getRange() method to get the counter cell. For a single cell, this method can take two variables the row and the column number. Here, we add in these two variables we collected earlier.

const counterCellRange = sheet.getRange(row, counterCol);

Now we need to add one to the current count in our counter cell. This means that we first need to collect the current value in the cell, which we do with the getValue() method.

We need a way to check if the value in the counter cell is empty. If it is then we want to give it a value of 1 otherwise we want to add 1 to the current value.

const counterCellVal = counterCellRange.getValue();

This can be achieved with a ternary operation. Think of a JavaScript ternary operator as a single line if statement in a way.

The ternary operator below states that if the value is less than zero (empty) then add one otherwise add one to the counter cell value.

const updateCounterCell = (counterCellVal < 0)? 1 : counterCellVal + 1;

Finally, we update the counter cell:

counterCellRange.setValue(updateCounterCell);

Done!

 

The following scripts will modify this code. We will touch on the changes only in the code discussion of each script.

Count Every Time The Cell Changes To the Selected Value

In this scenario, we only want to count when a cell changes to a specific value. For our example, this value is  “NEEDS EDIT”.

First off, we need to store a target value that we want to monitor for and if that value appears add one to our counter cell. Here on line 29, we set our target value to “NEEDS EDIT”. 

const tgtVal = "NEEDS EDIT"

You can change this to the value that you need for your project.

Next, we need to check if the cell that was edited was edited to the new target value before we update the counter cell.

To do this we first get the value of the selected range (Line 42):

const cellVal = range.getValue();

Then we create an if statement checking that the cell value matches the target value on line 43 and closes on line 54.
if(cellVal === tgtVal){

Apply The Script to Multiple Selected Google Sheets

What if we have different editorial teams or genres handled by different editors? We might want to put the data for each team in a different sheet tab to make it easy for monitoring. Here we will need to check all the selected sheets for any changes.

In our example now we want the “Team 1” and “Team 2”  sheet tabs. Both tabs are identical in format.

Monitoring multiple sheet tabs for changes to a cell value onEdit in Google Sheets with Apps Script

Let’s see how our code works now:

Two changes need to be made to the script here.

On line 25, we need to change the tgtSheet variable to tgtSheets and include an array of all the Google Sheet tabs we need to monitor with our Apps Script code.

const tgtSheets = ["Team 1", "Team 2"]; // The sheet names the data is on.

Then on line 38, we need to replace the first part of the if statement with a way that will look through our array of sheets and check if the current sheet contains the selected name.

This can be achieved with the JavaScript includes function. This function appends a selected array (In our case, the target sheets array) and takes a target value as an argument. Our target value is the select sheet that this being edited.

... tgtSheets.includes(sheet.getName()) ...

Get the Aggregate Count of the Changes

This final script checks all the cells in the target column in a single Google Sheet tab and checks if they have been changed to “NEEDS EDIT”. Then records the aggregate count of all the cells in a single counter cell in a separate sheet tab.

In our example, we will monitor the “Single  List” sheet tab for our desired change and record the total count of changes for all rows in the “Aggregate” sheet tab.

Calculate the aggregate of all changes to cells to the target value in Google Sheets With Apps Script

Check out the code:

First, we have replaced the counterCol variable with the counterCell variable and set that to cell A2 of the ‘Aggregate’ sheet tab.

const counterCell = 'Aggregate!A2'; // The column to update the counter.

Next, we don’t need to get the row number this time so we have deleted this variable.

Lastly, on line 44, we use the source property of the event (e) object to grab the current spreadsheet. This allows us to use the getRange() method to select our counter cell.

Conclusion

There should be enough samples there for you to work out your own combinations. If you want to test your new skills out, try to:

  1. Create an aggregate count of multiple sheet tabs.
  2. Move the counter column to another location in the sheet.
  3. Check for different target values in different sheets with different locations.

If you get stuck check out some of the other popular tutorials on using onEdit():

It’s always nice to hear how people use these scripts in their own projects and it can be inspiring to share your use cases with others. Feel free to share your experience and examples in the comments below.

Finally a big thanks to Mike Kanert for the inspiration for this post. Editor, writer, artist, actor, educator, polymath and all-around nice fella, you can find Mike on Instagram @Unremarkable_Us for his latest comic series made by parents for parents:

Unremarkable Us

If you have found the tutorial helpful, why not shout me a coffee ☕? I'd really appreciate it.

~Yagi

Sort and randomize data in Google Sheets with Apps Script

You might want to sort some Google Sheet data based on a condition in your Google Apps Script or simply sort by selected columns and rows after inserting a new row.

If you are always building sample data for tutorials like me, you might also be interested in randomising your data programmatically.

Fortunately, Google Apps Script’s SpreadsheetApp Class has built-in methods to handle just that for you. Plus it’s super easy to implement.

Check out the video tutorial below or grab one of the code snippets to add into your own project.

Continue reading “Sort and randomize data in Google Sheets with Apps Script”

%d bloggers like this: