If you didn't know, I collect records. Not "vinyls." No, please, never that.
I've been collecting since maybe late 2001 or around 2002, essentially
whenever I got into hardcore because that's what hardcore kids do. They
collect records. Well, at least some of us still do.
Also, if you didn't know, my profession is in web development. I guess you
could call me a software engineer, but that doesn't really sit well with me.
I don't know why. Maybe another blog for another time. Sometimes work isn't
very fulfilling. So I work on projects. Or try to. It's really hard to come
up with ideas. It's really not the idea that I'm after (most times) but the
learning experience.
Earlier this year, I came up with a project to mix those two things. Prior
to this, I've kinda had this idea for a "collection" app where you could
keep track of whatever collection of things you wanted. My initial thought
is that it would help me keep track of movies I owned (yes, I have purchased
multiple copies of movies I hadn't even opened). I guess it would have been
like a non-spreadsheet spreadsheet. I dunno.
I've also been wanting to keep track of my records somehow. For some reason,
I was hesistant to use the
Discogs. After some
time of not doing anything, I finally gave in and added
my collection. Mostly.
It's actually really fun and satisfying to just dig through my stuff. It's
also disappointing to find I don't have things I thought I did (miss you
Hope Con Cold Blue and first 7" :cry-emoji:).
On top of this, Discogs has an API, so that led to this: https://discogs.reallyxloco.com
The main thing I wanted to pick up when I started working on this was
CDK, AWS's Cloud Development Kit.
I'll try to sum up as best I can. Basically, if you want to orchestrate
your AWS resources, you would typically write some Cloudformation templates,
which aren't really too difficult to work with. CDK is kind of a wrapper
to Cloudformation, letting you create and deploy templates with
~~your~~Amazon's languages of choice, including Javascript, Python, and
Java. Besides just being able to build simple CF templates, you can
parameterize your deployments, like your DynamoDB table in your staging
environment can be
PAY_PER_REQUEST, but your production environment
should be
PROVISIONED. You wouldn't need to write two separate templates
for that.
Anyway, I wasn't trying to do anything fancy like that. I just wanted to
get familiar with CDK since we were starting to use it more at work. So,
my back end is two lambdas and a DynamoDB table.
The first lambda would read in my collection from the Discogs API and save
it to the table. Initially, it was just a normal flow—query the API,
get the data in the shape I wanted, then write to the table. Recently, just
for kicks, I refactored it to use
RxJS.
Why? Just because. This really isn't the right use-case, I think. It was
an interesting way to approach the problem though. The reader also makes
use of Step Functions. The Discogs API is borderline unusable because you're
limited to 60 requests per second. My collection was taking about 13ish
minutes to process 800+ records. Lambdas have a maximum execution time of
15 minutes, so not ideal. So the step function lets me run a chunk at a
time and just repeats until it's done.
The second lambda sits behind an API gateway, but it only reads from the
table. Really nothing fancy with this.
My CDK setup creates the DynamoDB table, deploys the lambdas, sets up
the step function, and sets up the API gateway. I originally also had
a Codepipeline going, but now I cut that part out (saves some money). I also
got around to adding some unit tests for the lambdas, so I think this
is a pretty solid system. I don't have a great way of updating my
collection; currently, I'm just destroying the table, doesn't sound
too bad, but that stack is coupled with the lambdas, so I essentially
have to tear everything down. Less than optimal for sure.
On the front end, it's just a React app. Nothing too fancy going on. I'm
using Material UI for the layout. It's pretty basic, but it works. I'm not
sure what else I want to do with it. I think this was my first serious
React project, and I'm happy with it.
Anyway, that's my show-and-tell.