EricDaugherty.com

Go Web Server using Lambda Function URLs

It has been possible to use Lambda Functions to host Go (Golang) Web Servers for years. I wrote about a project doing this back in 2016. More recently, the github.com/apex/gateway project provided a drop in replacement for net/http's ListenAndServe function that supported AWS REST APIs, and later the AWS HTTP APIs.

Amazon recently released the ability to access a Lambda Function directly via URL, without needing to configure the REST APIs or HTTP APIs. This remove an extra layer of configuration, complexity, and potentially cost, for deployments.

I created a fork of the github.com/apex/gateway at github.com/ericdaugherty/gateway that supports this new approach. A trivial example of usage in the README is:

package main

import (
	"fmt"
	"log"
	"net/http"
	"os"

	"github.com/ericdaugherty/gateway"
)

func main() {
	http.HandleFunc("/", hello)
	log.Fatal(gateway.ListenAndServe(":3000", nil))
}

func hello(w http.ResponseWriter, r *http.Request) {
	// example retrieving values from the api gateway proxy request context.
	requestContext, ok := gateway.RequestContext(r.Context())
	if !ok || requestContext.Authorizer["sub"] == nil {
		fmt.Fprint(w, "Hello World from Go")
		return
	}

	userID := requestContext.Authorizer["sub"].(string)
	fmt.Fprintf(w, "Hello %s from Go", userID)
}

This is effective and addresses the issue, but can make local development a challenge. I often replace the main implementation above with:

func main() {

	// check and see if we are running within AWS.
	aws := len(os.Getenv("AWS_REGION")) > 0

        http.HandleFunc("/", hello)

	// run using apex gateway on Lambda, or just plain net/http locally
	if aws {
		gateway.ListenAndServe(":8080", nil)
	} else {
		http.ListenAndServe(":8080", nil)
	}
}

This uses the net/http implementation of ListenAndServe locally during development, and the gateway library when deployed via Lambda. No configuration is needed, as the AWS_REGION environment variable is automatically set by AWS when running as a Lambda Function.

One limitation of the Lambda Function URLs is that you can only access the Lambda Function URL via the aws domain (ex: https://<function id>.lambda-url.<aws region>.on.aws/). One easy solution to this is to use AWS CloudFront. For a simple use case, create a SSL Certificate in AWS for your custom domain, and then create a CloudFront distribution, using the SSL Certificate and the AWS Function URL as the default origin. If you use this approach, you should also make sure to set the HTTP Cache-Control Header on your cache-able responses to improve performance and reduce your Lambda invocations.

You can use a more complicated CloudFront approach if your project uses a lot of static assets. The static assets can be deployed via S3 and the CloudFront distribution can pull from S3 or Lambda depending on the request path, but for smaller or low traffic deployment, everything can be served from the Lambda function.

From here you can build your application using any traditional tooling that is supported by the Go http/net library.

Package Delivery Detector

The following is an outline of how and why I built a tool to detect when packages are delivered to my front porch. If you have enough of the right infrastructure in place, this will serve as a guide to get it setup yourself. All the source code is available, so if your setup is different, you can take and modify what I've done to work for you.

This solution is built using Go, AWS, Google Vision, and Docker.

Motivation

In November I attended Amazon's AWS re:Invent conference to catch up on the current state of the cloud. I came away from the conference inspired to leverage an area that I didn't have a ton of experience, Machine Learning. I attended several SageMaker sessions and came up with a project idea:

A tool that would use Machine Learning to detect when a new package was delivered to my home.

I already had a lot of key infrastructure in place, including a Ubiquity UniFi camera that was pointed at my front porch. I also had a Synology Diskstation with Docker that I used to run both my UniFi controller and UniFi NVR.

This project took a long time. I kicked off work in early December and I just recently got to the point where I'm happy with the results.

Image Capture

The key to this project is to build a Machine Learning model that can determine if there is a package at my front door. To build this model, I needed a lot of input data, so I started with a small app that would take a snapshot from my camera every few minutes and save it.

Luckily, UniFi cameras make it easy to grab a JPEG of the current frame. This is what my front door looks like right now:

I was able to grab this image with a simple HTTP GET request to the camera's IP. Example: http://192.168.1.5/snap.jpeg

In order to enable this, I did have to turn on Anonymous Snapshots directly on the camera. Navigate to http://192.168.1.5/camera/config and click:

Since I would be running my image capture tool within my local network, I did not need to expose the camera to the Internet.

While this camera placement works well for my every-day use, it contains a lot of extra data that isn't relevant to detecting a package, so I needed to crop each image down to just the area where I thought packages might be placed. Here is the result of my crop:

I now had everything in place to build out an app to start capturing images. I wrote the app in Go (golang) as it is perfect for this type of systems programming. The app simply polls the camera every 10 minutes, grabs the image, crops it (if desired), and stores it in an S3 bucket or on the local file system.

The source code is available on GitHub.

I wrapped the app in a Docker container and deployed it on my Synology. The Docker image is available on Docker Hub. Here is a sample Docker run command:

docker run --restart always \
 -d \
 -e TZ='America/Denver' \
 -v /volume1/imagefetcher:/img \
 --name imagefetcher \
 ericdaugherty/imagefetcher \
 -imageURL http://192.168.1.5/snap.jpeg \
 -dir /img \
 -sleepHour 22 \
 -wakeHour 7 \
 -rect 0,400,900,1080

This creates and runs a new Docker container that captures the images, crops them, and stores them locally.

Since packages are generally not delivered in the middle of the night, and the night images are much harder to see, I decided to only run the image fetcher (and eventually the package detector) from 7a to 10p. So I added parameters to the ImageFetcher tool to stop capturing images overnight.

The -e TZ='America/Denver' parameter sets the timezone for the Docker container. This is important so that the timestamps are correct and the logic to sleep and wake work correctly.

The -v /volume1/imagefetcher:/img parameter maps the directory on the Synology to /img in the container, and then later the -dir /img specifies that the snapshots should be written to /img in the container, which will result in them being stored in /volume1/imagefetcher on the Synology.

If you would prefer to store the images on S3, you can add these parameters:

    -e AWS_ACCESS_KEY_ID='<Your Access Key ID' \
    -e AWS_SECRET_ACCESS_KEY='<Your Access Key Secret>' \

to the docker run command and this parameter:

    -s3Bucket <bucket name> \

to the imagefetcher command. You can then drop the -v /volume1/imagefetcher:/img and -dir /img, or keep them both and store the images twice!

Now you wait and capture data... I started with a month's worth of images before I trained the first model, but I continue to capture images and plan on training a new model with the larger set.

Machine Learning Model

You should now have a S3 bucket or directory full of JPEG images. Now comes the fun part, you need to manually sort the images into two different labels. I chose the highly descriptive 'package' and 'nopackage' labels.

I use a MacBook, so I used finder to quickly preview the images and run through them until I saw a change of state. Then I moved all the images into either the 'package' or 'nopackage' directory. Repeat until you've processed all of the images.

This is pretty labor intensive, but it did go faster than I expected it.

You should end up with two folders, named 'package' and 'nopackage'.

I then spent quite a while trying to figure out how to train the model using SageMaker. I found this pretty frustrating as I'm not really fluent in Python, and it turns out the Machine Learning space is pretty large and not super-obvious to pick up. Luckily, I came across a post from Jud Valeski where he was building a similar tool to determine when the wind blew the covers off his patio furniture. He used Google Vision for his solution, so I took a look.

As it turns out, using Google Vision to build a simple model is drop dead simple. I signed up for Google Cloud account, created a project, and then created a new Google Vision model. To create the model, I simply had to zip up the 2 directories and upload it. In about 10 minutes, I had a functional model!

Google also provides you a HTTP Endpoint that you can use to evaluate images against your model. You simply post a JSON body including your image base64 encoded, and it gives you back what label matches, along with its confidence level.

Package Detector

With the trained model in place and a public endpoint I can hit, all that was left was to build the final tool.

The final source code is available on GitHub. The Docker image is also available on Docker Hub.

I lifted much of the logic from the imagedetector to grab and crop the JPEG image. I then wrote new code to base64 encode the image and upload it to Google Vision. Based on the response, if a package is detected, an email is sent out to notify me.

The current version supports email as the notification tool. I leveraged Amazon's Simple Email Service (SES), but you can use any SMTP server you have appropriate access to.

This tool supports two triggers. The simple approach is to specify a simple interval, ex: -interval 5 and it will check every 5 minutes. However, I realized that the Unifi NVR is already doing motion detection, so if I could trigger based on that, it would only evaluate when there was a reason to do so. I came across a cool project by mzak on the Unifi Community Forums. Here is the GitHub Repo. Mzak realized that the NVR wrote a line to a log file (motion.log) every time motion was detected or ended. I leveraged his work to build a go library that would also monitor the log file. To use this, you must map the location of the motion.log file into the docker container. I do so with -v /volume1/docker/unifi-video/logs:/nvr and then point the packagedetector at this location with the -motionLog /nvr/motion.log parameter.

You can run the docker using the following command:

sudo docker run \
    --restart always \
    -d \
    -e TZ='America/Denver' \
    -v /volume1/packagedetector:/pd \
    -v /volume1/unifi-video/logs:/nvr \
    --name packagedetector \
    ericdaugherty/packagedetector:latest \
    -imageURL http://192.168.1.5/snap.jpeg \
    -rect 0,400,900,1080 \
    -motionLog /nvr/motion.log \
    -cameraID AABBCCDD1122 \
    -gAuthJSON /pd/google-json-auth-file.json \
    -gVisionURL https://automl.googleapis.com/v1beta1/projects/... \
    -sleepHour 22 \
    -wakeHour 7 \
    -emailFrom test@example.com \
    -emailTo test@example.com \
    -emailServer email-smtp.us-east-1.amazonaws.com \
    -emailUser  \
    -emailPass "" \
    -emailOnStart true

I now receive an email every time a new package is delivered!

Looking ahead, I'm interested in building in support for SMS or even push notifications, although I would also need to build an iOS app for that. I also plan on continuing to refine the model with additional images until I'm confident it will be correct nearly all the time.

Simple HipChat AddOn in Go

We use HipChat at work to stay connected. It works well and has some fun plugins, including Karma. I'll bet a few of you can guess what happens when you create a karma system in a company full of smart technical folks.

THEY GAME THE SYSTEM.

One of the more interesting hacks they discovered is to use the 'find and replace' functionality built into HipChat to secretly give (or more likely take) karma to/from another person. The proper usage of the Find/Replace functionality is something like:

> I can typo good.
> s/typo/type

The first line would be a typo that you made, and the second line allows you to 'fix' the typo. The key part here is that the second line modifies the first line and is not visible to any other users.

Who could let this fun discovery go unused? Not our team! Our chats were quickly filled with Karma Ninjas giving and taking karma from behind a thin veil of secrecy.

Luckily the Karma bot is open source. So I forked the source, made a fix and submitted a pull request right?

WRONG. Karma is written in JavaScript and writing JavaScript doesn't make me happy. So I decided to create a Karma Ninja bot instead to 'out' the folks attempting to be Karma Ninjas. Since I've been exploring Go on AWS Lambda recently, I figured this would be a great excuse to write more Go!

Starting Points

There are a few options out there already that really helped accelerate the effort.

First, the eawsy folks created a great shim for Lambda that allows you to deploy Go easily and it runs FAST. Their solution is much faster from a cold start than the other NodeJS shims out there.

Second, Nicola Paolucci has a great post on the Atlassian Blog about building a HipChat addon in Go.

Finally, there is a HipChat Addon project by David Jonas on GitHub.

The eawsy Lambda shim is being actively developed and the team was very responsive to a few of the issues/questions I had, including one that is apparently a bug in Go 1.8.

The Atlassian blog post does a great job of walking through how to build an add-on. This was the template I ended up using for my addon. The only note I have here is that Nicola uses the roomId to store the authentication token, but that only works for single room installations. I used the OAuthId field instead which seems to work for both single room and global installations.

David's project did not compile out of the box because of changes in the JWT Token upstream project. I thought about trying to fix is, but ultimately I wanted to understand how it all worked so I wrote mine without the template. I may go back and try to fix that project as an alternate implementation.

State

When I was envisioning building the plug-in, I assume I could build it statelessly. I assumed you could simply register a webhook with a regex and a callback URL that would return a message. But the webhook does not appear to let you return a message, you need to make a separate call into the HipChat API to post a message. Because of this, you need to maintain some state between calls.

EDIT: My first guess was correct, you can return a response to the webhook call and post a message. I ended up over-complicating it but the point was to learn more about AWS so it was still time well spent. Here is info on how to pass a response to the webhook.

I explored using both DynamoDB and S3 to maintain the needed state, which is really just an OAuth Token. They both worked fine but ultimately I chose to use S3 as it was the simpler and cheaper option for this use case.

Amazon does publish an AWS SDK in Go which made interacting with DynamoDB and S3 pretty easy. I do think the SDK is a bit 'low level' and could probably be made a lot easier to use with some convenience functions, but I'm just happy they support Go so I shouldn't complain.

The eawsy team does the necessary work to make the Lambda environment available from Go, meaning that you can simply use the Go SDK without any explicit authentication credentials or setup. By default, you simply inherit the IAM Role of the Lambda function. In this case, I simply needed to add permission to access DynamoDB and Se.

Code

The code itself is pretty straightforward. Like most go projects, I'm always impressed with how much you can get done in just a few lines of code across a handful of files. I published the project on GitHub: ericdaugherty/hipchat-karmacop.

AWS

This code does require some AWS setup. You need to define a Lambda function and build and upload the project using the Makefile (see the README for instructions).

You also need to setup an Amazon API Gateway entry to reference your AWS Lambda function. This will give you a public URL that your AddOn will reside at.

An S3 bucket is required to store the OAuth token used to post responses back to the HipChat API.

Finally, you need to add permission to read and write to S3 to the Lambda IAM Role configured for the Lambda Function.

That's It

This was a fun little project and shows a good use case for using Lambda to implement add-ons to cloud applications like HipChat.

How and Why I Became a 'Manager'

After my previous post about my approach to technical interviewing, I received some requests to write more about my career path. In this post, I'm will attempt to answer a question I get asked somewhat regularly: Why did I stop writing code and become a manager?

How did I choose Software?

Before we can dive into why I became a manager, we need to explore a bit more about who I was. Growing up, I loved Legos. I would spent hours building custom lego creations. As I got older, my family got a Commodore 64 and I spent hours playing cartridge games, watching Zaxxon fail to load from cassette tape, and typing in programs printed in computer magazines.

While I was in High School, I ran a dial-up BBS on a computer made from parts cobbled together from old computers and occasional purchases from Computer Shopper. I took all three classes in our computer lab at the High School: Typing, BASIC Programming, and Computer Drafting. Typing was the most important of those three. When I exhausted every class at the High School, I took a Turbo Pascal class at the local community college. If I wasn't already convinced (I was), after that I knew I wanted to write software for a living.

I went to the University of Illinois to study Computer Science. Studying Computer Science at UIUC was the fulfillment of a dream for me, but I quickly made a small change. During my Freshman year I switched to Computer Engineering. This was precipitated by my experience with the Introduction to Algorithms class. It involved way too much math and theoretical thinking. I just wanted to learn was how computers worked. The Computer Engineering program offered a lot more low-level classes and less theoretical programming classes. My favorite class was our microprocessor design, where we started with basic logic gates and built up to a pipelined processor, including writing a program and running it on the processor. My second favorite class was x86 Assembly programming. I was happy with my choice to switch to Computer Engineering, and it put me in the position I dreamed of as a kid, I got to get paid to write software.

Did I like Software?

After graduating I started working for a consulting company. They did a great job of training new college graduates with a well-defined self-directed training course in C++. That was, practically speaking, the first and last time I got paid to write C++. The world was quickly transitioning to the web and Java. I was among the first members of our staff to learn Java for new projects, and being a primary Microsoft shop, a little J++ as well.

It was amazing. I was learning something new every day, and getting to build applications that real people were using every day. It was also when I first learned to dislike process and management.

This is probably best exemplified by a quick story. On the first project I had an opportunity to lead, I was asked to document the object model. According to our process, this involved opening up Microsoft Word, and defining each class and public method, including method signatures and JavaDoc formatted documentation in Microsoft Word. I thought it was a complete waste of time, I was frustrated that management didn't understand software development, and I just wanted all the 'useless' people to get out of the way and let me build software.

I had another experience that furthered my distain for anyone other than developers. We were implementing Enterprise Application Integration (EAI) software written by Active Software for one of our clients. This involved moving data between several different enterprise systems. The pattern we used was to define a single message format for each data type that would contain all the data that could be consumed by every end systems. Active Software called this a 'Canonical Message'. However, apparently the client didn't understand what the world 'canonical' meant, and I spent the better part of an afternoon and the entire evening in a conference room with the entire project team attempting to come up with a new way to describe a canonical message without using the word canonical. It drove me CRAZY. I wanted everyone to get out of my way and let me go build something.

I swore I would never do anything but write code.

What Changed?

Fast forward a few years, and I had built up a solid skill-set, primarily in Enterprise Java but also a solid understanding in C#.Net. I learned how to build websites in Oracle Application Server 1.0, proprietary frameworks, Enterprise Java, Spring, and other variations.

I loved learning new frameworks. Each one brought solutions to problems I had struggled with, and as I learned, a new set of problems.

And ultimately, this was the primary factor in my evolution. After a while, a new framework brought a quick review and shrug instead of excitement and a deep examination.

My focus over this period was building enterprise applications running inside large companies. These are important systems and 'fun' to work on, but at the end of the day are largely reading data from a database, displaying it in HTML, capturing some changes, and writing data back to the database. It got old, especially when I became somewhat disillusioned with the evolution of the frameworks.

During this time, I also got to spend a lot more time working directly with clients. While this is often a frustrating experience, it allowed me to learn about a lot of different companies, industries, and business models. I came to respect the challenges that they each faced and the solutions that they had developed.

These factors caused me to pick my head up and look around at the world I lived in, and wonder what else I could be doing.

What Did I Know?

After spending close to a decade working primarily for consulting companies, I had broad but shallow knowledge of a lot of industries. I had worked for Fast Food, Manufacturing, Insurance, HR, Logistics, Banking, Financial Exchanges, and Retailers. I knew a little about a lot, but I didn't have deep knowledge of any. I realized that what I really understood was the consulting industry.

This worked out well since I currently worked for a consulting company. I looked around I started seeing other problems I could solve. Luckily, I was part of a fast-growing company with the trust of the ownership.

It started small. We had a small project and needed a part time PM. I stepped as the technical architect, but I also worked with the client to make sure we were delivering what they needed, and keeping them up to date on the project status and the challenges.

I also learned how the owner interviewed and I started helping out as the company was growing quickly and he was doing more interviews than he could handle alone.

I built trust and over time I took on more responsibility.

How Did I Learn?

I often get asked how I learned how to manage a company. The answer is pretty simple: just like I learned how to code. I read a lot and learned through trial-and-error. I also had great mentors.

This is an example conversation I often had with the owner of one of the companies:

Me: We should be doing X.

Him: Why?

Me: Because <a lot of reasons I thought made sense>.

Him: No.

Me: Why?

Him: Because <a lot of reasons I had not thought about>.

I would take what he said and reformulate my pitch based on this new information. It would often take several iterations before I either got a yes or I became convinced my idea wouldn't work for our company.

The great thing about the owner was that nearly every yes was followed by "go do this, I'm holding you accountable". This was a great way to grow my responsibility and take ownership of my ideas and see them through.

This conversation happened a lot, and over time I really understood what he cared about, and what his overall philosophy was. I agreed with much of his philosophy, but not all of it. We had many spirited discussions over the years, and I came away with a well developed philosophy of my own.

During this time, I expanded my reading habits beyond technology. I started reading business books, articles and anything and everything that taught me about how businesses worked. I learned about public companies by reading their 10K filings. I took a deeper look at our clients and tried to learn why they made decisions that I had previously thought were... strange.

And That Was That?

My transition to 'manager' wasn't over. My family decided to relocate to Colorado. Starting over in a new city with almost no professional network, I chose to go back to senior technical positions where I was actively writing code. I still have a passion for building software and, luckily enough, skills that were still valuable. But I eventually grew restless and started looking for a way to leverage my broader skills.

Then a very important event in my story happened: I took the family to Disney World. No, I didn't have an epiphany on my tenth ride on "It's A Small World", we only rode it 3 times. Instead, I met the CEO of Xcellent Creations on the flight home and found an opportunity to leverage the expertise I'd developed over the years to help grow a small mobile shop into an industry leader that we eventually sold to WPP.

Ultimately, I enjoy solving problems and building things. Whether those are Legos, software, or organizations, it doesn't really matter. What matters is that it that the problems are challenging and that I'm working with great people.

My hope is that the organization that we've built today does for our employees what my past companies did for me. I want to help our team expand their skills, take on new challenges, grow as individuals, and view the world differently. And along the way, ship great apps.

Technical Interviewing and Hiring

There has been some discussion in the technical community recently about the use of algorithms and coding tests during the interview process. Here is a sample:

Hello, my name is David. I would fail to write bubble sort on a whiteboard. I look code up on the internet all the time. I don't do riddles.
— DHH (@dhh) February 21, 2017

Hi, I'm Py. Used to think I didn't need to learn CS. I was lazy. I practiced for interviews, and it made me a better Software Engineer. https://t.co/yD5NzcmTwF
— Py ⚔ (@Piwai) February 23, 2017

I have my own thoughts I would like to share, but this isn't really a topic for 140 characters.

Why should you care what I think?

I've spent a lot of time interviewing and hiring developers across several organizations. For a while I was doing interviews every day. In fact, I remember telling my recruiter that he couldn't schedule more than 2 interviews in a day because by the third interview I found I was no longer effective. I estimate that I've interviewed over 300 candidates, and been the primary decision maker on over half of those. I have been a key member of the interviewing and/or hiring team in two custom software services companies that have grown from 30 or fewer people to over 115 during my tenure. I've done this a lot, and I was there to see my successes and my failures.

I spent over a decade primarily as a software developer. I've written production code in LabView, C++, J++, Java, VB, Perl, C#/.Net, JavaScript, Flex, Scala, Objective-C, and probably more. I've probably written more Java than the rest combined, but I was always learning something new and during the time that I was doing these interviews, I knew what I was talking about. I would like to think I still very technically competent, although many of my current employees enjoy proving me wrong regularly.

I leaned this approach from a great mentor early on, and adopted and extended it over the years. We all stand on the shoulders of giants.

Is this the one true way?

Before I describe my approach, I want to focus on a key point lost among the 140 character conversations. Your interview process should identify candidates that will work in YOUR organization. Your company probably isn't just like David Heinemeier Hansson's, or anyone else you read on twitter. It probably isn't just like the companies I've worked for either, so if you blindly copy what was successful for me, you will probably still fail.

I've spent most of my career working for services companies building great software, from enterprise back-end systems in Java and .Net to amazing mobile applications in Objective-C, Swift, Java, and Kotlin. Because these companies are services companies, I expect my employees to be able to interact directly with out clients. The people I hire must be capable of more than just writing great code, they must solve our clients problems, by understanding what they really need, communicating effectively with technical and non-technical people alike, and delivering a great technical solutions.

I also expect my employees to be able to learn a new technology quickly. The landscape is always changing, and when you are building software for other companies, you do not have the luxury to define what technologies are and are not acceptable.

Therefore, I focus on hiring great problem solvers. If you are not a good problem solver, you will not work on my teams, no matter how amazing you are at language/framework/platform XYZ.

So what is the interview like?

Most of these interviews last about an hour. The four sections are...

Cool Stuff - ~15 Minutes

I start all the interviews by asking the candidate what projects they think are cool or projects they are proud of. These can be work projects, open-source projects, school projects, or just tinkering around and learning projects. This engages the candidate and allows them to talk about topics they are comfortable with.

This tells me several things. First, it tells me whether they communicate effectively. I actually prefer when they have a project in an area that I am not knowledgeable. Selfishly, it lets me learn about something new, but the primary reason is that it show me whether they can explain a technical concept to me that they understand well. If it is in an area that I know well, it lets me probe to see how well they really understand the topic.

It also helps me level-set where the candidate is. Ignoring their resume, the projects and accomplishments they are proud of tell a very clear story about their skill level and world view.

Coding Problems - ~20 Minutes

What!?! Yes, I have two basic coding/problem solving problems that I make each candidate work through, on paper, in front of me.

Problem 1:

This is essentially a recursive coding problem. You can solve it in any OO programing language with a basic knowledge of the core syntax. It does not require any API/Framework specific knowledge. The right answer is about 4 lines of code.

I have given this problem to everyone from college students looking for their first job to senior architects. They should all be able to solve it. I do adjust my expectations based on the level indicated by their resume and 'Cool Stuff' answers.

Achieving a Passing mark on this doesn't necessarily require a perfect answer right away. I'm happy to ask questions, and give small hints to the candidates if they get stuck. This is actually one of the most valuable parts of the process because it allows me to see how well they listen, and how they think through a problem. But if you can't write (simple) code down on paper with a reasonable level of accuracy, and you can't write a basic recursive method with a few hints, you are probably not going to cut it.

Problem 2:

This is where I date myself somewhat. Problem #2 is ideally writing a SQL statement, but is really a boolean logic problem. It does not require anything other than very basic SQL syntax. No Joins, or anything even slightly fancy. The right answer would fit in a single tweet.

Again, I have given this problem to everyone from college students looking for their first job to senior architects. They should all be able to solve it. I have run into college students that didn't know SQL, and in those cases I was able to adapt it to be a boolean logic problem that they could solve in general set notation.

Again, success here is showing me how you think through and solve problems. You don't need to write the correct answer down flawlessly the first time, but you have to show me that you can think through a problem, listen to feedback/hints if necessary and incorporate them into your thinking.

Alternate Problem:

Depending on time and level, another question I've asked A LOT, mostly to Java and .Net candidates, is how garbage collection works in the JVM (or on .Net).

Occasionally I'll get a candidate that actually knows the answer, and while impressive, this is actually somewhat disappointing. The point of this question is to have them show me their problem solving skills applied to a real technology that they depend on every day, but probably don't think about.

This questions shows me how well a candidate can listen, how well do they think on their feet, and how well do they actually understand computer science concepts.

Again, the 'right answer' isn't necessarily the point. Having a great conversation with me where in the end they really grasp the concept and feel like they are a better developer makes both me and the candidate feel good about the interview.

Side Note: I was working with a startup (not my full time job) and helping them make their first full time developer hire. I used these problems and rejected a candidate. The CTO, who had not actively developed in several years, challenged me about the questions, so I gave him the interview. He passed both in less than 10 minutes, and realized that he didn't want anyone on his team that couldn't solve these problems. The next person we interviewed passed, was hired, and did a great job for them for several years.

Resume Due Diligence ~ 15 Minutes

Once the problems are solved, I will use the next section to tackle any areas of interest that I think are appropriate. I will often read through the resume and ask questions about specific projects/accomplishments to determine how accurate the descriptions are and what role the candidate actually had on those projects.

If they profess deep knowledge in a certain language/framework/platform, I may go deep in this area to see how much of an expert they really are. I love learning something new from candidates, and they usually enjoy teaching the interviewer something new as well!

A big take-away from this section is: how honest are they about their resume? Were they exaggerating, or underselling themselves?

I can also use this time to follow up on areas the candidate showed a particular interest in to see how deep their knowledge is.

Candidate Questions ~ 10 Minutes

The final section is an opportunity for the candidate to ask me questions, about the role, culture, expectations, etc. While this is a key part in making sure the candidate is sold on working for me, their questions are also a window into their thought process and outlook, and can be informative in making a hire/no-hire decision.

Ultimately thought, this section is about closing the candidate. Regardless of whether you will hire the candidate, you want them to be sold on your company, and you want them to feel good about the experience. Even if you pass on them, you want them to leave the interview with a positive opinion of you and your company. It is a small world.

Why do I do is this way?

I've covered a lot of the why along the way, but it important to reiterate and expand on my motivations and goals.

First, I want to reiterate that I target hiring great technical consultants. We build software solutions for other companies, and the skills we look for reflect that. This is probably not a great approach for other types of companies.

I firmly believe that in order to be a great software consultant, you have to be a great problem solver. To me, this includes listening and understanding the problem (solving the right problem), deep enough technical expertise to identify a good solution (there is rarely one RIGHT solution), the ability to communicate what the solution is, what the trade-offs are (there are always trade-offs) and how it addresses the actual problem, and the ability to deliver the solution.

In the companies that I worked for, we certainly didn't expect an Associate Developer to interact directly with the client and execute all of these steps, but this is still the ultimate process and everyone should be able to participate at the level of their experience.

I also don't believe in team interviewing for smaller services companies. A consultant working for one of these companies will need to be successful across different clients and different teams. I believe in the accountability of a single decision maker, and if the process is consistent, then the team members know the people getting hired went through the same process they did. Ultimately, if the team trusts the hiring manager, they will trust the candidate that they hire. However, this doesn't scale and as a company grows, the process needs to evolve.

Again, this probably doesn't work for other types of companies, and team interviews make sense in a lot of cases.

Finally, I do it this way because it works. Yes, I made a few bad hires over the years. In each case, I went back to look at the interview notes to see what I missed and how I could do better next time. In some cases I was being too optimistic because I was desperate to fill a need. In some cases the candidate simply snowed me. In some cases, there is an issue that I can't reasonably expect to uncover in an interview.

Final Notes

The lifeblood of the interview/hiring process is the great work that your recruiters do. I firmly believe in having in house recruiters and I've been blessed to work with some great ones. You know who you are, thank you.

I have no idea how many false negatives I've had. I would argue it is unknowable. But the growth of both companies would suggest that I didn't turn away too many qualified candidates.

These opinions are my own, and do not reflect those of my employer. I am no longer in a role where I make any hiring decisions on technical candidates, so don't expect that reading this post will give you an inside edge. ;)

This post is too long. Unfortunately, I'm not a good enough writer to write a shorter post. I'm sorry.

GoLang Alexa SDK

I continue to be interested in Alexa, Amazon AWS Lambda, and Go (golang), and I've found a new way to deploy Alexa apps in Go on Lambda.

While Go is still not an officially supported language on Amazon Lambda, there are several ways to make it work:

Lambda_proc - Node.js wrapper for your go binary
GopherJS - Cross-Compile your Go code to NodeJS
eawsy Lamba Go Shim - Python shim for your go binary

In my previous project where I built a screen scraper in Go and deployed it to Lambda, I utilized Lambda_proc. This does work pretty well an is a solid solution.

I found GopherJS to be an interesting project, but a pretty indirect way to get Go onto Lambda. Lambda can run Go Linux binaries, so converting your Go to JavaScript seems like the wrong approach.

The eawsy team released their new tool earlier this year and it seems to be the cleanest approach. The overhead between the Python shim and your Go code is clean and fast. They utilize a Docker container to build the necessary bridges from Python to Go, resulting in very fast execution. You can write log messages to the Lambda console using log.Printf. And it is FAST. The HelloWorld skill runs in sub-millisecond times!

To build Alexa skills in Go on Lambda, we still need an SDK. Amazon publishes SDKs for Java and JavaScript but not for Go. So I ported the Java SDK into Go as ericdaugherty/alexa-skills-kit-golang While this does not (yet) have the full depth of the Amazon SDK, it does enable you to build simple Alexa skills using Go and deploy them on Lambda.

Take a look at the alexa-skills-kit-golang project for usage and samples, and give the eawsy Lamba Go Shim a try for your next Go project on Amazon AWS Lambda.

- Updated 3/4/2017 - updated run time based on newer stable eawsy shim speeds.

Lamda, Go, and GotSport via API Gateway

While I've been primarily living in the mobile world recently, I've been intrigued by a the developments in a few other areas of technology.

Go, or golang, has been increasingly becoming my preferred language for side projects. I really like the concise nature of the language, the ability to deploy on multiple platforms, and the short tool chain. It 'just works', and is fun to program in.

AWS continues to add interesting services and features. I recently moved my website from GoDaddy to host on S3. This really started my thinking about living in a 'serverless world'. While practically speaking hosting on S3 isn't really all that different than virtual hosting at GoDaddy, it is really just scratching the surface. You can now build pretty interesting applications without running a server (virtual or otherwise).

I begin to think about how you could build a full application using the AWS stack without an EC2 instance. Of course, a ton of thought has already been put into this. The Serverless tool allows you to easily configure Amazon to use the API Gateway with Lambda to deploy functional APIs. There is also a website dedicated to living serverless.

I've also been working on a few Amazon Alexa applications for the Echo, which also uses Lambda as the preferred deployment program.

So I thought it was time to build something that actually works.

My son plays soccer and the Colorado Soccer Association uses the event.gotsport.com website to post schedules and results. So I built a screen scraper in Go to parse his schedule into JSON so I could use it in my Amazon Alexa application. (The scraper is here: https://github.com/ericdaugherty/gotsport-scraper) But I hard-coded the resulting JSON into that application.

I figured I could build a general API that could be used by anyone to convert the schedules into JSON. And I could easily deploy it on Lambda.

In order to get Go running on Lambda, you need to 'work around' the fact that it isn't officially supported. The lambda_proc library (https://github.com/jasonmoo/lambda_proc) does just that. It uses a node.js wrapper that invokes your Go application within the Lambda runtime. The repository has a good example that should you how to write and deploy a go app on Lambda.

From there, I just needed a simple Go app to take the input JSON, run the gotsport-scraper I wrote, and return the resulting JSON.

The final step was exposing the Lambda function as an HTTP API. This is where the API Gateway comes in. It allows you to specify an endpoint and method to trigger the Lambda function. The basics are pretty straightforward. You define a resource (/gotsport) and a method (GET), and map it to your Lambda function. However, the tricky part is the mapping of the HTTP Request to the Lambda function, and the result of the Lambda function to the HTTP Response.

Here is the full lifecycle:

I decided to use Query String parameters to pass data to the function, so you could just cut/paste from the URL you wanted converted. You can define the query strings in the Method Request section, but this just seems to allow you to use them when testing and isn't required. I did have to map the query parameters into the Lambda, so I created an application/json mapping (since the Lambda function consumes JSON). The mapping function is:

{
    "eventId" : "$input.params('EventID')",
    "groupId" : "$input.params('GroupID')",
    "gender"  : "$input.params('Gender')",
    "age"     : "$input.params('Age')"
}

This maps the Query String Parameters using their names (again, as used on gotsport.com so you can cut and paste) into JSON values that match those used by me gotsport-scraper tool. This is then passed to the Lambda function.

The Lambda function runs, fetching the requested URL, scraping it, and returning a JSON value. However, the lambda_proc function returns both an error value and a data value containing the results of the Lambda function. I wanted to map the output to just contain the JSON representing the schedule. So in the Integration Response step in the lifecycle, I used the application/json mapping function:

#set($inputRoot = $input.path('$'))
$input.json('$.data')

This just extracts the data element from the JSON returned from the Lambda function and passes it back as the HTTP Response Body.

The proper approach in the Integration Response step is to use a RegEx to determine if an error occurred or not, and return the proper HTTP response code and appropriate body. For now I'm assuming a 200 response with valid data.

That's it! I now have an HTML->JSON screen scraper for the GotSport website deployed as an API.

Want to see how this all works. Here are the resources:

lambda_proc - Run GoLang apps on Lambda
gotsport-scraper - Screen Scrape the event.gotsport.com site
gotsport-lambda - Lamba function to execute gotsport-scraper
Amazon API Gateway - General API Gateway documentation
Amazon API Gateway Mapping - Documentation about writing mapping files.

Want to test it out? Grab a specific schedule from the CSA Youth Advanced League 2016 site. Then cut/paste the query string parameters onto my API URL: https://j4p9lh1dlb.execute-api.us-east-1.amazonaws.com/prod/gotsport For example, the U-11 Boys Super League would be: https://j4p9lh1dlb.execute-api.us-east-1.amazonaws.com/prod/gotsport?EventID=46461&GroupID=511424&Gender=Boys&Age=11