I have not used Redshift for the past few years, having migrated from AWS Redshift to Google BigQuery. However, Redshift Serverless, which is a pay-as-you-go Redshift, finally appeared and I started to get curious about it. Sure enough, I …

I wanted to create a React site with an authentication function easily, but I gave up because it was too much trouble to build a backend each time. However, we tried AWS Amplify to see if we could easily build a backend. Since I have tried…

When you add a Lambda Function in AWS Amplify, a template is generated in JavaScript, so this is a memo on how to turn it into TypeScript. The following document was used as a reference https://docs.amplify.aws/cli/function/build-options p…

I had hoped to be able to create a quick SPA site on the spur of the moment, but I had given up on React because it is too big a job to create even a small site. However, I recently started touching Recoil for React, and with Recoil, React…

I previously tried to handwrite an AWS CloudFormation template and failed.... www.ekwbtblog.com What I learned at that time was that CloudFormation templates are not the kind that can be written by hand, but rather generated using some kin…

It's been a while since I installed the AWS CLI. I see that it is now at version 2. Once you set it up, you don't have a chance to touch it and forget about it, so here is a memo on how to set it up for yourself. Basic Structure Set profil…

Periodically, AWS S3 files are copied to Google Cloud Storage. There are many ways to copy, but here we use the gsutil command for the Google Cloud SDK, The Google Cloud SDK uses a Docker image of Alpine Linux. However, at some point, I st…

In Google Cloud Storage, I tried to get a list of files in a subdirectory and process those files. However, when I looked at the error message that the file was wrong, I found that the list of files retrieved included a subdirectory. Reaso…

I received an email from Google Cloud Platform. The Node.js version of the function I originally created for Gooble Cloud Functions was version 6, and this time the version 6 service was ending, so the email was a request for an upgrade. I…

I have tried AWS Glue before and here are my thoughts on it. AWS Glue is made of Apache Spark, and it was interesting for me to touch Spark for the first time at that time, I was thinking that I would put together a usage guide for Glue wh…

How to access AWS S3 from Spark (Google Dataproc). procedure Spark Configuration The following Spark and Haddop settings will allow you to read and write AWS S3 files from Spark. Load the following AWS-related jar files into Spark aws-java…

This is a working memo for a simple Slack Bot using AWS Lambda and Serverless Framework. Creating a Slack Bot is a laborious process with many steps, but with AWS Lambda, you don't have to worry about the server environment. In addition, u…

As I wrote in AWS CloudFormation Articles, AWS Lambda and API Gateway are managed in Serverless. Here is a poem about how we got there and a brief explanation of how to use Serverless. impetus What URL the Lambda is called with is configur…

About this Article This section describes how to configure the API Gateway authorizer to be able to CORS on the Serverless Framework. The details are transcribed below, with an outline added. https://serverless.com/blog/cors-api-gateway-su…

About this Article Describes a setting to allow the authorizer to be used by multiple Lambdas when deploying Lambdas using the Serverless Framework. procedure Include resultTtlInSeconds: 0 in all http events that use the authorizer, as in …

We originally used AWS Redshift, but it was quite expensive, so we switched to Google BigQuery. After that, Redshift Spectrum came out, and I was wondering if it might be better than BigQuery, so I touched it and here are my impressions. A…

As the title suggests, this is my impression of AWS CloudFormation. impetus When calling AWS Lambda from a URL, you have to use the AWS API Gateway GUI to do the mapping, such as "call this Lambda for this method of this URL," but this is …

There are times when you want to retrieve S3 files from one account to another. In such cases, it is quicker to create a dedicated user who can download the S3 file, but this can also be done by granting access to the destination S3 to the…

In the following directory structure, how to get a list of directories directly under a bucket or "dir_A1/dir_B1" in JSON. s3://bucket_1/ dir_A1/ dir_B1/ dir_C1/ dir_C2/ dir_C3/ dir_A2/ dir_A3/ When directly under the bucket $ aws s3api li…

I was building a single page application and did not want to publish it externally, but wanted to put it on a non-local server to test delivery, so here are my notes on building it. Elements Since it was a static distribution and S3 seemed…