I was working on a program related to GCP and wanted to store some data. It was a one-off piece of data that was not large enough to set up a database, nor was it likely to write a join query between the data. To begin with, setting up a d…

Periodically, AWS S3 files are copied to Google Cloud Storage. There are many ways to copy, but here we use the gsutil command for the Google Cloud SDK, The Google Cloud SDK uses a Docker image of Alpine Linux. However, at some point, I st…

When loading Google Cloud Storage files into Google BigQuery, multiple files can be loaded at once. However, I got stuck trying to do that with Google BigQuery's Node.js SDK, so here are my notes on that. Overview and Issues Load using tab…

In Google Cloud Storage, I tried to get a list of files in a subdirectory and process those files. However, when I looked at the error message that the file was wrong, I found that the list of files retrieved included a subdirectory. Reaso…

We previously used the GCP Pyhton library to get a list of Google Cloud Storage subdirectories, and this is the Node.js version of that. Python version is here www.ekwbtblog.com procedure The GCP Node.js SDK documentation is kind enough to…

I wanted to get a list of Google Cloud Storage subdirectories using the GCP Python library, I got stuck, so here's a note on how and why. Libraries in other languages are API wrappers as well as Python libraries, so I think they can be app…