Thoughts

Tag: Learn to Code

Build an Image Search Microservice with the Imgur API

It’s been a while since I’ve had time to touch Free Code Camp backend projects, but during my week off I decided to build out the next API project I had yet to attempt, the Image Search Abstraction Layer microservice. I love building and working with APIs, so this was a fun project to build. Let’s get started!

First, let’s initialize our git repository and jump into it.
git init image-search-microservice
cd image-search-microservice

Now, we’ll want to setup our project’s package.json. If you haven’t tested out Yarn yet, I highly recommend giving it a try.
yarn init

Speaking of which, make sure you’ve updated your copy of Node for this tutorial, as I’ll be using some ES6.

Whether you’re using npm or yarn, you can skip through most of the default settings. Personally though, I’m changing my entry point to app.js.

Next, let’s set up our .gitignore to ignore the node_modules folder and our .env file, which we’ll introduce a little later. Drop these two lines in your terminal to do this quickly.
echo node_modules >> .gitignore
echo .env >> .gitignore

Now let’s install the node modules we’ll need for this project. The list is pretty short for this one:
yarn add express mongoose request
yarn add dotenv --dev

Go ahead and create your app.js and setup the basic scaffolding for an express app.

const express = require('express');

const app = express();

const port = process.env.PORT || 3000;
const server = app.listen(port, function() {
    console.log(`Server listening on port ${port}`);
});

At this point, you should be able to run nodemon (assuming you have it installed globally) in your terminal and see your server running on port 3000. If not, go ahead and compare your project files.

I’m all about writing modular code, so we’re going to set up some folders and files like so:
config/db.js
models/history.js
routes/index.js
services/imgur.js

First we’ll set up the routing. Add const routes = require('./routes/index');
and app.use('/', routes); in our app.js to be used as middleware. Then, setup your routes/index.js like so:

const express = require('express');
const router = express.Router();

router.get('/', (req, res) => {
  res.send('Hello');
});

router.get('/latest', (req, res) => {
});

router.get('/search/:q', (req, res) => {
});

module.exports = router;

Now, we’ve moved the job of routing out of our app.js. The advantage to doing this is that if our app were to grow, we could add additional route files and keep our project modular and clean. I’ve also setup the basic routing our application will need: a root endpoint where we can eventually display a landing page if we so choose, a /latest endpoint where our app will display the ten latest searches, and a /search/:q endpoint where we can pass a string in to search.

Next, head over to the services/imgur.js file and drop this in:

const request = require('request');

exports.getImage = function(search, page = 1) {
  return new Promise((resolve, reject) => {
  });
};

We’ll be making a request to the Imgur API through our app, and the request module makes that really simple. Also, we’re going to be calling this function back in our route file, so I’ve set it up as an exported function that takes two parameters. The first is the search term, the second is the pagination option which we can default to ‘1’ if nothing is passed. And finally, I’m a big fan of promises over callbacks, so I’ve set this function to return a promise.

Now, per the documentation, the request module can be invoked with two parameters – an options object and a callback. In our options object, we’ll need to pass in our unique client ID from Imgur. Swing over to api.imgur.com if you haven’t done so already and register your application. You can choose ‘Anonymous usage without user authorization’ when prompted.

Now that we’ve got our code, we can build out an options object within our promise statement like so:

let options = {
      url: `https://api.imgur.com/3/gallery/search/${page}?q=${search}`,
      headers: { Authorization: 'Client-ID kfbr392kfbr392' },
      json: true,
    };

In here, we’re setting up the URL that we’ll be connecting to Imgur with. Here, we’ve used ES6 template strings to cleanly drop in the page and search parameters from our getImage function. Next, we set the headers as requested by Imgur to allow proper authorization of our API call (but, make sure to replace my ‘Client-ID with your own). And finally, we’ve specified that we want our response to be in JSON format.

Next, we’ll build out our callback. First, we can build a function called getPics that will take three parameters: error, response, and body. I’ll then make a quick error check inside the body of the function that ensures no errors occurred and the response received from Imgur has a status code of 200. Then, we want to take the body that was returned, and filter out items in the array that are albums. We’ll do this to best mirror the demo app shared by Free Code Camp, as we’ll want to provide a response with both a direct image link and a link to the image’s context – and unfortunately this wouldn’t work well if we included albums.

Then, after we’ve filtered out albums, we then want to map over the response and cut out all the extra information we’re not using. All we want to return is a url, a snippet, and the context. Finally, we’re going to resolve our promise with our newly transformed data, jump out of the callback and then setup our request function with its two newly created parameters. In the end, it will look like this:

function getPics(err, response, body) {
  if (!err && response.statusCode == 200) {
    body = body.data.filter(image => {
      if (!image.is_album) {
        return image;
      }
    }).map(image => {
      return {
        url: image.link,
        snippet: image.title,
        context: `https://imgur.com/${image.id}`
      };
    });
    resolve(body)
  }
}

We’re almost ready to test it out, but first we have to jump back into our routes file and make sure to require our new imgur service file const imgur = require('../services/imgur');
and to call it in our /search/:q endpoint.

Here, we’ll pass the ‘q’ parameters as the search string of the function, and then we’ll let our second parameter req.query.offset handle the optional ?offset=42 flag. Then, once that promise returns, we’re going to output the response to our browser as JSON.

imgur.getImage(req.params.q, req.query.offset).then(ans => {
  res.json(ans);
})

Now fire up nodemon and give it a test. If all is working so far, you should be able to search Imgur by pointing your browser to localhost:3000/search/puppies. If not, do some debugging or a quick code comparison at this point.

Now that this piece is working, all that we have left to add is the history component. I personally enjoy working with mLab in development just as much as in production, so I’m going to proceed with this tutorial as such. If you’d prefer to run MongoDB on your local machine prior to deployment, or you want to learn more about how to get setup on mLab – checkout my previous tutorial on building a URL shortener for more information.

The first thing we’re going to want to do is setup our environment variables. This way, we’ll able to connect to our mLab database in either production or development, without worrying about sharing our credentials on Github. In your app.js file, drop in the following at the very top:

if (process.env.NODE_ENV !== 'production') {
  require('dotenv').config();
}

Here, we’re setting up our server to require the dotenv package if we’re not running in a production environment. Later when we push this up to Heroku, their platform will automatically set an environment variable of NODE_ENV as ‘production’, but on our development machines – we won’t worry about that.

Next you’ll need to create a .env file in the root of your project. This file will then assign your environment variables every time you spin up your server. In that, drop in the following as it relates to the mLab database you set up:

DB_HOST=usernamehere:passwordhere@ds1337.mlab.com:1337
DB_NAME=databasenamehere

Now that we have our environment variables setup, we can dive into setting up our connection to the database. Back in your app.js, you’ll want to require your /config/db.js file with a const db = require(‘./config/db’);

Then, jump into your config/db.js file and drop in the following:

const connection = `mongodb://${process.env.DB_HOST}/${process.env.DB_NAME}`;
const mongoose = require('mongoose');
mongoose.Promise = global.Promise;

exports.db = mongoose.connect(connection);

Here, we are setting up a connection to our mongo database and then using mongoose as an ORM for interacting with it. Also, I’ll be using some promises, which were recently deprecated in mongoose, so I’ve set the native promise library to be used instead. And finally, we export the connection for use elsewhere in our application.

Next, in our models/history.js file, I want to set up a schema for our database. First, we’ll need to require an instance of mongoose. Next, we want to setup a schema for our database. This way, when we pass data into the model later in our routes, we’ll be able to let MongoDB know what we want each new document to look like, and specifically in this case – to always attach a timestamp to each new entry. After that, we export our model for use.

const mongoose = require('mongoose');

const historySchema = new mongoose.Schema({
 term: String,
 when: { type: Date, default: Date.now }
});

const History = mongoose.model('History', historySchema);
module.exports = History;

At this point, our database is setup and we’re ready to start building out the queries we’ll need.

First, let’s make sure to require our new model in our router with const History = require('../models/history');

Now, we want to add a new entry into our database every time a search is made. Right before we send back a response with the results, drop in the following to add that search query into the database: new History({ term: req.params.q }).save();

And finally, we need our /latest route to return the most recent 10 entries. For that, we’re going to query the database like so:

History.find({}, 'term when -_id').sort('-when').limit(10).then(results => {
  res.json(results);
});

Here, we first pass in an empty object, which will return all documents. Then, we specify that in those documents, we only are interested in the search term and the date, and we’d like to specifically exclude the unique _id field from the results. Afterwards, we instruct the query to sort the results in descending order and to only return the ten most recent documents. Finally, we pass those results through a promise and return them as JSON.

Go ahead and run your app with Nodemon and verify that everything is working. If so, you’re ready for deployment. Since we’re using ES6, you’ll need to define your node engine in your package.json, otherwise Heroku will, at the time of this writing, default to version 5.1.1 which won’t support ES6 unless you use strict mode. Also, you’ll need to make sure to add a start script to your package.json. To see what those look like, check out the file on my GitHub. And finally, make sure to drop your environment variables into your Heroku app, otherwise you definitely won’t be able to connect to your database.

That’s all there is to it. Here’s my final code for the project. Personally I went back and threw in pug as a template engine and a favicon server. The final deployed version can be seen here. Feel free to leave any questions or comments below.

Promises – What are they and how do I use them?

While Javascript is single threaded, we as humans are not. We have the ability to process multiple requests at the same time. This enables us to efficiently go about our days without letting minor obstacles stop our progress. If we see a pothole in the road, we shift to the side to avoid it. So how can we take that same aspect and apply it to our projects? Therein lies the beauty of asynchronous code.

In the past, Javascript developers had to rely primarily on callbacks or additional libraries to east the pain of “callback hell” when making external API requests. And, while callbacks may still have a place in your codebase when making local requests or awaiting a function to finish executing – wouldn’t it be great if we had a more structured and reliable way of retrieving and processing data asynchronously from an external source? We do, and they’re called promises.

To keep things neat, I’ll show some examples of how to utilize promises using the jQuery.ajax() method. In recent versions of jQuery, the objects returned by this method have an implementation of the promise interface and will make these methods a bit easier to show off. If desired though, a native asynchronous request could be wrapped in a new Promise() function with similar results.

Let’s dive right in and make an AJAX GET request. For illustration purposes, I’ll use the GitHub API results for my account.

let alertMessage;
let getGithub = $.ajax({
  method: 'GET',
  url: 'https://api.github.com/users/leftynaut',
  success: function (response) {
    alertMessage = response.location;
  }
});

console.log('Location: ' + alertMessage);

Now, we’ve already made our first asynchronous request and we have our getGithub variable referencing that response. But, before we made that call, we set created a variable called alertMessage, and then after we made the request, we asked for the console to log what should be my current location of ‘Austin, TX’. So is that what’s going to log to the console? Not without a promise it won’t.

Once again, our AJAX call is an asynchronous request. So although the console.log function is called below it in our code, that function isn’t going to wait to execute while our app is making a request to the GitHub API. So it will run before our alertMessage variable has been defined, and we’ll see Location: undefined logged to the console.

So how can we wait until our location has been passed into the alertMessage variable before we call an alert? With a promise. Our getGithub variable has a promise method attached to it called .then(). This method is going to accept two arguments, one callback function which will resolve on a successful call, and an (optional) one for a rejected call. The beauty here is that whether your call succeeds or fails – we’re waiting for the response before we continue.

Speaking of calls, a phone call is a great way to visualize this. Imagine a phone call between yourself and a friend, Roger, where important information is discussed. You want to call another friend, Shirley, but you can’t until you’ve received all of the pertinent information from Roger and you’ve disconnected with him. After this occurs, you are then able to call Shirley – but not a moment sooner. Promises work similarly – they will wait until the data has fully resolved (or failed) before they proceed.

So if we were to use a .then() method on our previous example, it would look like this:

getGithub.then(function(){
  console.log("Location: " + alertMessage);
});

Now, we would successfully get the console to display the location we were hoping for. If we wanted, we could pass in a second parameter to our .then() in the form of another callback function to catch any errors thrown by our original getGithub promise function.

Where promises really shine though is in their ability to chain promises. This way, you can ensure that asynchronous calls have the needed info before they proceed to the next call, which then will ensure it has the needed info, and so on…

Let’s see an example of this in action:

var githubAPI = 'https://api.github.com/users/leftynaut';

function getGithubLocation(githubProfile) {
  return $.ajax({
    method: 'GET',
    url: githubProfile
  });
}

function getCoordinates(location) {
  let coordinatesAPI = `http://free.gisgraphy.com/geocoding/geocode?address=${location.location}&format=JSON&from=1&to=10&indent=false`;
  return $.ajax({
    method: 'GET',
    url: coordinatesAPI,
    dataType: 'jsonp'
  });
}

function getSunrise(coordinates) {
  let sunriseAPI = `http://api.sunrise-sunset.org/json?lat=${coordinates.result[0].lat}&lng=${coordinates.result[0].lng}`;
  return $.ajax({
    method: 'GET',
    url: sunriseAPI
  });
}

function consoleSunrise(data) {
  console.log(data.results.sunrise);
}

getGithubLocation(githubAPI)
  .then(getCoordinates)
  .then(getSunrise)
  .then(consoleSunrise)

In this example, we’re making three asynchronous AJAX requests. First, we’re creating a function to get my GitHub data. Next, we create a function that passes my location of ‘Austin, TX’ to an API which will convert that string to GPS coordinates. After that, we create a function that will take those coordinates, and pass them to an API which will return the sunrise information for that location. And finally, we pass that string into a function that will print the time to the console. Thanks to promise chaining, we’re then able to link all of those functions together in a neatly structured call.

Goodbye Callback Hell.

Time Complexity and Logarithmic Bar Tricks

In the past couple of days, we have dived deep into data structures and time complexity. Learning the intricacies of computer science has been a rewarding challenge – and has resulted in quite a few “aha” moments. My personal favorite thus far has been binary search trees – especially realizing just how quickly a computer can find a value using an algorithm based on logarithmic complexity. But, even more interesting was realizing that a human can pull that off too. Let’s call it a logarithmic bar trick.

  1. Tell someone you can guess a number between 1 and 1,000,000 in twenty or less guesses, provided they let you know whether your guess is high or low
  2. Use an algorithm of logarithmic complexity O(log n) to guess
  3. ?????
  4. Profit

How does that work exactly? Every time you guess, you’re breaking down the original sample size by 50%. So your first guess is going to be 500,000. They tell you it’s higher. Your second guess is 750,000. They tell you it’s higher. At this point you’ve only made two guesses and you’ve already eliminated 75% of the possibilities. Keep up with this method, and you’ll have their number in twenty guesses tops.

Let’s see how that would look in Javascript.

var min = 1;
var max = 1000000;
var guess;
var number = Math.floor(Math.random() * max) + min;
var counter = 0;

while (guess !== number){
        counter += 1;
	guess = Math.floor((min + max) / 2);
	console.log('Guess #' + counter + ':', guess);
	if (number > guess){
		min = guess + 1;
		console.log('Number is higher');
	} else if (number < guess) {
		max = guess - 1;
		console.log('Number is lower');
	}
}

console.log('Number:',number);

First, we set our min and max variables. Then, we'll create a variable to hold our guesses called guess. Next, we'll set a number variable to a random integer between our min and max. And lastly, I want to keep track of how many guesses it's going to take to find the number, so let's create a counter variable and set it to 0.

Then, we build a while loop which will continue to run until our guess variable is equal to our number variable.

Inside of our loop, we first increment our counter variable. Next, we set the guess variable equal to to the average of the min and the max combined.

Next, we setup an if statement that's going to check whether our guess variable is higher or lower than the number. If our guess is too high, we're going to set our max variable to one less than the current guess. If it's too low, we'll set the mix variable to one more than the current guess. That's all there is to it.

For fun, I went ahead and wrapped this in a function, dropped the function call in a for loop that ran 10,000 times, and pushed the counter results to an object after each run. When only running it once, I rarely would see the algorithm find the number in less than 16 guesses. But, when running it 10,000 times, I was able to catch the program guessing correctly on the first try at least once.

Want to up the ante? Tell your friend you can guess a number between 1 and 1,000,000,000 in thirty or less guesses. Thirty? Yup. Just take your maximum number and continually divide by 2 until you hit one or less. The amount of times it takes you to get there is equal to the maximum amount of guesses needed.

1,000,000,000 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 / 2 = 0.93

Helpful links:

Moving the Needle

While my experience with Free Code Camp, Treehouse, and countless other virtual learning platforms have provided me with a great foundation in web development, I decided that I wanted to stop permitting this passion to only thrive in my spare time. Thus, I have chosen a path that allows me live, eat, and breathe software engineering all day, every day, for the rest of the year. I have recently joined cohort #47 at Hack Reactor ATX (formerly MakerSquare). Currently, I am only a few hours into day three and here’s what we’ve already covered:

  • Scopes
  • Closures
  • “This”
  • Value vs. Reference
  • Debugging
  • Data Structures (Stack & Queue)
  • Prototype Chains
  • Complexity Analysis

It has already been a real challenge during some pieces, but the opportunities to learn from those bumps in the road have already been tremendous.

Copyright © 2017 Thoughts

Theme by Anders NorenUp ↑