How to Copy to a Google Storage Bucket with GitHub Actions (GCP)
In this article, I will show you how to copy a file to a Google Storage Bucket using GitHub Actions.
These instructions were written and tested on a Mac.
Step 1. Create a project folder
Make a project folder and change to it using these commands:
mkdir -p ~/projects/actions/gh-action-zip-gcp-bucket
cd ~/projects/actions/gh-action-zip-gcp-bucket
Step 2. Make a GitHub workflows folder
To make a GitHub workflows folder, do the following in the root of the repo:
mkdir -p .github/workflows
Step 3. Make a source folder in the project
To make a source folder, do the following in the repo:
mkdir -p source
Step 4. Create the files
Create empty files to fill in later by doing the following in the root of the project:
touch README.md
touch LICENSE
touch .github/workflows/upload-to-gcp.yml
touch source/bacon.txt
You can name the source file (bacon.txt
) anything you want. If you want to follow my example, generate some text from https://baconipsum.com/ and paste it into source/bacon.txt
.
Just leave the folder name as source
- because a later step will reference that specific folder name.
You can do a search for "top ipsum generators" for alternatives.
Step 5. Edit upload-to-gcp.yml
- Open
.github/workflows/upload-to-gcp.yml
in VS Code - Paste the following into the file and save it:
name: Zip and Upload to GCP Bucket
env:
TARGET_ZIP: source.zip
BUCKET_FOLDER: uploads
on:
push:
tags:
- 'v*'
jobs:
deploy:
runs-on: ubuntu-22.04
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Zip the folder
run: zip -r ./${{ env.TARGET_ZIP }} source/
- name: Authorize GCP
uses: 'google-github-actions/auth@v2'
with:
credentials_json: ${{ secrets.GCP_SA_KEY }}
# Step to Authenticate with GCP
- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud@v2
with:
version: '>= 363.0.0'
project_id: ${{ secrets.GCP_PROJECT_ID }}
# Step to Configure Docker to use the gcloud command-line tool as a credential helper
- name: Configure Docker
run: |-
gcloud auth configure-docker
# Step to Upload the file to GCP Bucket
- name: Upload to Google Cloud Storage
run: |-
TARGET_ZIP=${{ env.TARGET_ZIP }}
BUCKET_PATH=${{ secrets.GCP_TARGET_BUCKET }}/${{ env.BUCKET_FOLDER }}
EXTENSION="${TARGET_ZIP##*.}"
FILENAME="${TARGET_ZIP%.*}"
TIMESTAMP=$(date +%Y-%m-%d_%H-%M-%S) # Format: YYYY-MM-DD_HH-MM-SS
LATEST_FILENAME="${FILENAME}_latest.${EXTENSION}"
NEW_FILENAME="${FILENAME}_${TIMESTAMP}.${EXTENSION}"
gsutil cp ./$TARGET_ZIP gs://${BUCKET_PATH}/${LATEST_FILENAME}
gsutil cp ./$TARGET_ZIP gs://${BUCKET_PATH}/${NEW_FILENAME}
The workflow does the following:
- Triggers when a tag beginning with v is pushed (v1.0.11 for example)
- Checks out the files
- Authorizes using a GCP Service Account pasted in the repos Secrets section
- Sets up gcloud
- Authorizes with Docker - even though Docker isn't used directly in this example
- Builds two filenames (latest and one with a datestamp in the name)
- Copies the zip file of the source folder to the bucket under the two new file names
- The latest file will be overwritten with the latest copy
You can comment out the second gsutil cp
line if you don't want to store previous versions.
The drawback to storing previous versions is that they can build up over time. You would need to periodically remove them. Or create some automation to only maintain a certain number of previous versions. That's beyond the scope of this article.
Step 6. Setup Google Cloud Platform (GCP)
Create a new project id
- Browse to the Google Cloud Platform console
- In GCP create a new project for demo purposes
Create a service account (SA)
- See my article: How to Setup Google Cloud Credentials for Automation (GitHub Actions, GCP, Service Account)
- Be sure to assign the SA with enough rights to create files in a bucket
Create a new bucket
- Create a new bucket (in the GCP console under Cloud Storage)
- Make sure the SA has access to it
Step 7. Fill in the secret vars
Be sure to set the secret vars in the repo for the following:
- GCP_SA_KEY
- GCP_PROJECT_ID
- GCP_TARGET_BUCKET
For a little extra security, I like to store the project id and bucket name as secret vars. Partially because this is going into a publicly accessible demo.
I would at least put them in vars and not hard code them in the middle of your script.
For documentation purposes for a private repo it would probably be convenient to have them in environment vars at the top of your file. So when you are looking at the workflow in 6 months you can easily see and remember where it's storing things.
You should always make the SA key a secret.
Step 8. Trigger the build
- Get the current tags:
git tag
- Increment based on the last tag
- Trigger the build
Here is an example:
git tag v1.0.1
git push origin --tags
Step 9. Monitor the workflow actions
- In the repo, click on the Actions tab
- The run should turn green to indicate a pass
Step 10. Check the bucket
Check the bucket for updates and / or new files.
- Browse to Buckets - Cloud Storage
You may need to refresh the browser window
A note about billing
Remember that GCP may give you credits, but it's not completely free.
Always keep an eye on billing and shutdown your experiments when not needed to keep costs low.
You can monitor project costs here:
Sample project
Here is a sample project:
Conclusion
In this article you learned how to copy a file to a Google Storage Bucket using GitHub Actions.