Ever been faced with an issue where you can’t move your column to the far left in order to Vlookup? I just came across this issue recently.
The Problem – Vlookup
I needed to add teachers to a list of students who had to move into their class for two tests because their teacher (Hanna John-Kamen) would be absent on the test day.
Essentially, I was splitting the students into the remainder of the classes for that session time. In this student list sheet (Hannah 302-15) I had allocated the students new rooms and each room had a new teacher.
I couldn’t change the teacher proctors test list sheet ( Q2U1 Teacher) because the teachers who reference it would get confused. Unfortunately the teachers name I wanted was in column B and the reference (The Speaking classroom) was in column I.
I need to put my first Vlookup in M2 of the Hannah 302-15 sheet. If I tried to Vlookup it might look like this: =VLOOKUP(L2,‘Q2U1 Teacher’!B:I,–8,False). Where L2 is the class I am searching for in the Hannah 302-15 sheet. ‘Q2U1 Teacher’!B:I is the range where B is the name column and I is the Class number. –8 is clearly wrong. Vlookup will not accept a negative reference (It’s stupid, I know). False: because the list was not sorted.
OpenPyXl is a Python open library that allows you to read and write Microsoft Excel files. Specifically, the ‘*.xlsx’ file extension. It helps you to create programs to create and modify files and automate your processes in excel.
NOTE: This post requires that you have some knowledge of Python and the OpenPyXl library. The library also needs to be installed for you to use.
Quite often, I find that I have to work with ranges of data that I need to either copy and paste into a new file or files, or copy > modify > paste into files.
The OpenPyXl library allows you to look at every cell of a file and either copy it or modify it by using the openpyxl.worksheet.Worksheet.cell() method. This method allow you to access each cell by the row and column as a numerical value.
Note! Unlike everything else in coding, rows and columns start with one(1) and not zero(0).
To select whole ranges of our data we need to iterate through it by both row and column and then store that data in a list to be pasted to the new file, spreadsheet or location that we desire.
The following example will take you through the process. For your own data you will need to modify the file, sheet and range locations. Everything else should be good to go.
You can find the whole code at the end of the post.
Why does your MS Excel look weird?
To be honest my screenshots of the ‘.xlsx files will be in Libreoffice. But this simple example will be able to load without issue in MS Excel.
The source data for this example is a very simplified version of grade data that I work with day-to-day. The goal of our program is to simply copy the Section 12 results into a new file. The file for this spreadsheet is called: GradeSample.xlsx. I have put it in the same folder as my Python program.
For the purpose of this example we have a file that we want to save this data into: Section12Grades.xlsx. We could just as easily use a template and save the file under a different name – for example template.xlsx could save as sec12Grade.xlsx. This could be useful if I wanted to save a file for each section.
The receiving file looks like this:
Loading the Source Data and Receiving File
Okay, let’s get started with the program. First we need to load both the source data and the receiving file.
Loading the Spreadsheets
#! Python 3
# - Copy and Paste Ranges using OpenPyXl library
#Prepare the spreadsheets to copy from and paste too.
#File to be copied
wb=openpyxl.load_workbook("GradeSample.xlsx")#Add file name
sheet=wb.get_sheet_by_name("Grades")#Add Sheet name
#File to be pasted into
template=openpyxl.load_workbook("Section12Grades.xlsx")#Add file name
temp_sheet=template.get_sheet_by_name("Sheet1")#Add Sheet name
We import he OpenPyXl library first.
Next, we’ll open the source data with wb = openpyxl.load_workbook("GradeSample.xlsx") . Once we have loaded the workbook we need to tell Python which sheet tab we want it to work in. We do this by calling the workbook (wb) and then get the sheet by it’s name: sheet = wb.get_sheet_by_name("Grades")
We repeat this step with the receiving data file that we want to paste our Section 12 grades into.
Copying the Section 12 data
Looking at the section 12 data, we want Python to be able to copy from column A2 to D14. the OpenPyXl .cell() method takes a number which is much easier to iterate, rather than the column letters. So let’s look at that range again:
From: A2 is now column = 1 and row = 2
To: D14 in now column = 4 and row = 14
Once we access this data, we need somewhere to store it before we paste it into the new file. We will use a nested list for this.
Copy Range Data
#Copy range of cells as a nested list
#Takes: start cell, end cell, and sheet you want to copy from.
#Adds the RowSelected List and nests inside the rangeSelected
In line 3 we create our function copyRange. It contains 5 arguments that we could add with our information as follows:
startCol = 1
startRow = 2
endCol = 4
endRow = 14
sheet = sheet ( The name of the sheet variable we are copying the data from which is GradeSample.xlsxsheet Grades)
In line 4 we create an empty list called rangeSelected this list will have data from every row we have selected.
Line 5 starts the for loop through each rows. Each row contains data in each column so we create an empty list (rowSelected) here in preparation to add the column data to in the next for loop (Line 6).
Line 6 loops through each item in each column of the selected row. Line 7 then adds the data from each column to the rowSelected lists.
Once it finishes this loop, it adds the data from the rowSelected lists into the rangeSelected lists. It then moves down to the next row and repeats the process continuing through the rows until the loop meets it’s end at row 14.
Finally the copyRange function returns the rangeSelected list to be used at our discretion.
Pasting the selected data
Now we have a nested list of all the rows and the information in the columns we require from those rows.
We will use that list add it to our Section12Grades.xlsx in Sheet1.
We want to add the data starting at row 3 this time because we have a title on row 1 and column headers on row 2.
Line 3 starts our pasteRange function and contains the following arugments:
startCol = 1
startRow = 3 (because we want to paste the data 1 row down.)
endCol = 4
endRow = 15 (because we want to paste the data 1 row down.)
sheetReceiving = temp_sheet (This is the variable for Section12Grades.xlsx with the sheet name, Sheet1.
copiedData = rangeSelected ( The returned list from your copyRange function)
Line 5 creates a count number variable starting with zero(0) so we can start looking through our copiedData lists by rows from the beginning.
Line 6 begins the row loop like the copyRange function.
Line 7 adds another count number variable so we can loop through our copiedData list by columns.
Line 8 begins the column loop.
Line 9 adds the copiedData cell-by-cell to the new file. It does not save it here but holds it in memory to be saved to the file we choose in the future.
Finally, we add 1 to each of the counters so we can move onto the next cell.
Running a Copy and Paste as a function
We are now going to create a function to copy the data using the copyRange function and paste the data using the pasteRange function and then we will save the Section12Grades.xlsx file contained in the variable, template.
Spacer is a Free Google Add-On that creates row spaces by column categories.
I use it as an academic administrator to create a space or two at the end of each class section to create a visual break between sections. This makes it easier for my staff to quickly identify class sections.
Allow you to select a column with the categories you want to use to space your data.
Set the height of your header – so that is not spaced too.
Set the thickness of your spaces.
Set the color of your spaces
Set starting and end position of your color filler.
How to use Spacer
Before You Start:
Before you start, create a Google Sheet with the data in it you want to space.
Select All the data.
Go to the menu bar and select: Data>Sort Range... and select the column you want to sort by. This will be the column you will use to space your data by the categories in that column.
In a Google Sheet, select the Add-Ons menu.
Select: Spacer>Create Spaces
A sidebar will appear to the right.
Choose the column you want to use to Space your data.
Choose how high your header is.
Chose how thick or deep your want your header to be.
If you want a color, select a color that you want to add to your spaces.
Select where you want to start the color and end the color.
If you are not happy with the color, space, color width or made a mistake, simply click: UNDO
I keep getting the following popup:
This is because the starting value of your Color Width is greater than your End value:
I have a list of people with their favorite color and fruit. Let’s say I want to sort the Google Sheet by favorite color (Fav Color) and put a space at the end of each color category to make it easier to read.
My data looks like this:
To space this data we must do the following:
Select all the data: ctrl+shift + a
Sort the data by column B. Select: Data> Sort range
Click the: Data has a header row tick box. Select the column: Fav Color. Then click the Sort button.
Your data is now sorted by favorite color:
Time to use the Spacer. Select: Add-ons > Spacer > Create spaces
Spacer will open at Sidebar on the right:
We want to space our data by Fav Color which is column B. For Column, select: B
We have a single Row Header so we can leave the header selection at 1:
We want to give our spaces a thickness of 2 rows:
Next, we choose a color. Let’s choose purple. A color picker will pop up (the picker may vary depending on your system) select the color and click: OK:
This is the result:
Now choose where to start and finish the color fill for the space. I’ll start it at column A and end it at column E.
You sure can. Google has a fully supported script editor that you integrate with your Sheets, Docs, Forms, Slide, Gmail, Calendar and pretty much every aspect of the Google Suite. It’s name: Google Apps Script.
Google Apps Script allows you to do all sorts of things like building short code to modify sheets and docs, create macros, develop add-ons mess around with gmail and so much more.
You know, I’m pretty confident in my use of Google Sheets. I use a lot of it’s advanced features and formulas in my daily work.
But one thing was bugging me. Every time I created a formula and then grabbed and dragged (or double clicked) that little box in the bottom right of the cell to have it repeat, the cells would change.
This normally is great, because it moves by a cell each time and makes my life easier.
What if I want to keep a certain cell constant in a formula?
Enter the Absolute Reference.
The absolute reference allows you to lock either or both aspect of the cell. That is it can lock the column and/or the row. This is done by simply adding a “$” before the column or row. Here is how we do it:
$A$1 – locks both Column and Row
$A1 – locks only the Column
A$1 – locks only the Row
Awesome!!! Now you can drag that little square wherever you want and whatever part of that cell you locked wont change while everything else will.
F4 – The Shortcut From the Gods
So, this is all well and good, but it still means that after I have finished entering the whole formula, I have to go back into the cell and change the cell value with that dollar sign “$”.
F4 to the rescue. You can change the value live here by cycling through $A$1, $A1 and A$1 before continuing with your formula. While completing a formula simply,
click a cell or range you want to get a value from and you want to lock or make an Absolute Reference.
Press <F4> until you get the desired Absolute Reference combination.
Continue with the rest of your formula.
Example – My Fruit Binge
I really love fruit (Okay, I’m ambivalent about fruit, but I am trying to sell the example here!)
I have a list of fruit that I have eaten in the last hour. My wife has caught me with a pile of cores and skins scattered around me and wants me to count how much this food binge cost me in order to make me feel bad.
Of course, I immediately run to my laptop and bring up my list of fruit I just ate. I can also remember how much each item costs (Yep, the example is breaking here, stay with me).
To work out the total costs of what I ate, I will count how many of each item and multiply that by the costs of each item.
Here is my Google Sheet so far:
Lets plug in the formula to count the total number of Bananas.
Things look good so far. In cell G3 I have run countifand selected the range C2:C13. And I only want it to count if the cell contains the text “Banana” which I indicate by cell E3. Finally, in cell H3 I multiply the total number of times Banana appears (G3) by the cost of the item (F3).
The end results:
Just for shits and giggles, lets see how that countif formula will work if I drag it down with that cheeky little box in the bottom right of the cell.
As you can see things have gone a bit Pear-shaped (nailed it!) here. We can see 2 pairs in the list in column C but the formula is only counting one in cell G7.
Let’s take a look at the formula in G7.
We can see here that the range has changed from C2:C13 to C6:C17. That is not helpful at all. We need to make these into an Absolute Reference. The fruit criterion also moved, but we wanted it to come down to read Pear so that is pear-fict (Not so good that time).
Let’s fix this up. We need to change the countif range in cell G3 to an Absolute Reference from C2:C13 to $C$2:$C$13. This is kinda awkward so let’s try that <F4> shortcut out and put in the formula again fresh. To do this I did the following steps.
selected the range C2:c13
hit the magical <F4> button once and it created: $C$2:$C$13
entered a comma: ,
selected the range E3
closed the bracket: )
Looks good. We’ll grab that little box down the bottom again and drag (or double click) it down and see if we get that extra pear.
Boom! It worked. We now see 2 pears counted. Check out that bottom cell. It’s lock in the range value and only changes the fruit criterion.
Just to be pear-dantic (Hey! That was okay!) let’s see how guilty we should be for our fruit binge.