Coding Part II: Radfems

 

Hi there!  Read Part I of my coding journey here and get the chrome extension here!

Part of what reignited my desire to code was a project I did in December 2016 called He to She.  It was a simple chrome extension that changed instances of masculine pronouns (he, his, him) to feminine ones (she, hers, her).  It was pretty janky (code is here) and was largely written with the help of tutorials.

But it was SO much fun to write!

And I wanted to do more.

For my next project, I wanted to create a chrome extension that brought up a new woman in STEM with each new tab.  There are plenty of new tab extensions out there and I figured it should be within my realm of understanding.

Nope!

It was way beyond my capabilities.  So I headed back to the drawing board with CS50.  This time, I was determined to finish.  I had projects to create!

Granted, it wasn’t a straight shot.  I got distracted and built a shelf and a robot, but I got through the course in the end.

So there I was with a blank sublime text page, an idea for a project, and no idea how to start.

Sketching out the plan

I began by sketching out the user experience of my project:

  1. User opens a new tab.  A page with a random woman to know in STEM appears.  There is a picture, her name, and a short bio pulled from Wikipedia.  At the bottom of the page, there is a link to read more on Wikipedia.

Then, I broke down the actions I would need to figure out:

  1. Get the name of a woman in STEM
    1. I decided to put a pre-determined list of names in a txt file since I couldn’t find a good RSS feed or other such list of women in STEM
  2. Get info on the woman from Wikipedia
  3. Get picture of the woman from Wikipedia
  4. Add Wikipedia information to the new tab page
  5. Add some sort of backup option if the user is offline or Wikipedia doesn’t work
  6. Format the page so it looks pretty!

Using the Wikipedia API

I decided to tackle the info from Wikipedia first.  This was, in my mind, the meat of the program and so it was important to get it working.

To do this, I knew I would need to get a JSON object from Wikipedia.  JSON (JavaScript Object Notation) is basically an easily readable bundle of information from a website.

To get a JSON object, you need to request it.  One way to do this is to use JQuery to request a JSON object from an API.

Side note: We had used JQuery in CS50, so I understood the basic concepts, but I hadn’t sat down and read the documentation.  Like I said in Part I, I like getting context then going back for the foundation.  For me, it’s much easier to write a program using JQuery then go back and read the documentation than to read the documentation top to bottom and sit down to write a program.

CS50 is set up in to facilitate that style of learning.  They give you serious training wheels so you don’t really need to understand what you’re writing.  It’s up to you to follow up and read documentation to truly understand what you’re doing.

Back to my project: I started looking into whether Wikipedia had an API and found this page.  I also found this page about parsing JSON responses from JQuery and this one about Wikipedia extracts.  I used the URL from the Wikipedia extract page and some of the code from the parsing JSON page to put together a first working version of my javascript code.  A screenshot of my first successful grab of Wikipedia info below!

Screen Shot 2017-04-14 at 9.59.41 AM.png

link to v1 js code on Github

Seeing the words appear on the page was IMMENSELY satisfying, but I was a long way from a launch ready extension.

Making my JSON request dynamic

My next step to make my JSON request dynamic.  In v1, I had hard-coded in a single URL that retrieved the summary info for the wikipedia page for “Weather.”  I looked around for how I might build the wikipedia request and had an a-ha moment after re-reading the JQuery documentation on .getJSON.

I realized that “Data that is sent to the server is appended to the URL as a query string.”  This meant I could pass a number of parameters into “data” as an object, and the output would be url.com?A=B&X=Y for the object {A: B, X: Y}.

So I looked again at my wikipedia URL and translated the query string into a data object like so:

https://en.wikipedia.org/w/api.php?format=json&action=query&prop=extracts&exintro=&explaintext=&titles=Weather&redirects=1
{ "format" : "json",
  "action" : "query"
  // etc etc etc
}

You’ll notice that the empty parameters like exintro=& are linked to empty strings in my data object like this –> {“exintro” : “”}

I also (later on) read on the Wikipedia API documentation page that you can (and should) pass multiple variables to the same parameter using the pipe symbol “|”.

For example, prop=extracts&prop=pageimages became:

{ "prop" : "extracts|pageimages" }

My final query looked like this:

 var wikiAPI = "https://en.wikipedia.org/w/api.php?";
 var test = $.getJSON( wikiAPI, {
 format: "json",
 action: "query",
 prop: "extracts|pageimages",
 exintro: "",        // this is to get just the intro paragraph
 titles: fem,        // variable with name of the random woman to lookup
 piprop: "original", // this is to get the image
 redirects: "true"   // this returns the final page after any redirects

Loading in my ladies

Now that that was done I had to figure out how to load in a list of ladies, and retrieve a random lady to look up.  In CS50, we had used separate text files to hold long lists so I decided that’s what I would do.  I created a .txt file and manually typed in a list of women in STEM.  (The list is basically a mashup of various buzzfeed/wikipedia/other lists, with some women who’s bios are super short removed, so suggestions/edits welcome!).

To do this, I would need to use a GET request.  This is exactly what I had done with the Wikipedia request, so this would be easy!  Also, since my file was local and I didn’t need to send any additional parameters, my GET request was VERY straightforward.  Just $.get(filename).  See below for my code:

 $.get("radfems.txt")
   .done(function(femlist) {
   var femlist = femlist.split(/[\n,]+/);
   console.log("getFemList success");
   randomFem(femlist, callback);
   })
   .fail(function() {
   console.log("getFemList error");
   })
   .always(function() {
   console.log("getFemList complete");
   })

You’ll see there are no additional parameters passed to my GET request outside of the filename.

The next piece was to parse the returned object.  Basically, I got back a big string, and I wanted to split it into an array of names.  Unfortunately, I had formatted my text file so that each woman was separated by both a comma and a newline.  I didn’t want to go back and reformat, so I decided to use a regular expression (regex) to match the comma+newline pattern.

Using an online regex tester, I tested out my regex and got back an array of my women’s names.  Success!

Retrieving a name

Now that I had my list of women loaded in, I needed a way to get a random name to then lookup in Wikipedia.  This part was very straightforward.  I used the built in Math.random() utility to generate a random number, and then used that number as the index to get a woman from my array.

var random = Math.floor((Math.random() * femlist.length));
 var fem = femlist[random];
 while (fem == "") {
 fem = femlist[random];
 };
 getFemInfo(fem, callback);

Local/backup image

The last piece of the puzzle was having some sort of backup image or text for when the user was offline.  I googled around and found this women in STEM bio series jewelry designer aubergdesigns had created.  I pulled a few of these images to use as my backups.

I put these images into their own “imgs” folder, and then linked to a random image in that local folder whenever the Wikipedia request failed.

 var test = $.getJSON( wikiAPI, {
    .... // this part is shown above...trying to keep it concise!
 })

 .done(function(test) {
   .... // this part is really long, see github for code
.fail(function() {
 console.log("getFemInfo error");
 getLocalFem(callback);
 })

// LOCAL FILE RETRIEVAL FUNCTION
var getLocalFem = function(callback) {
 console.log("getLocalFem");
 var random = Math.floor((Math.random() * 3));

 var img = "/imgs/" + random + ".jpg";
 callback("", "", img, "");
}

Callback functions

Up until this point, I had been using a global object container (more here) to store a number of variables – such as the name of the woman I was looking up and the Wikipedia page ID – that I wanted to access from several functions.

However, I had read that it’s generally discouraged to use global variables.  I think using a global object container is less discouraged, but still I wanted to see if there was another way.

Much googling led me to the concept of callback functions.  I wanted certain functions not to execute until the data they needed was ready, so I thought it made sense to use callback functions.  Essentially, a callback function is a function that is passed to another function as an argument.  The callback function is then called/executed WITHIN the other function.

I like examples so I googled around some more and found this code for displaying a quote in each new tab which helped me to understand how callbacks are used in the wild.  (I often find it helpful to look at longer pieces of code in Github, as examples in documentation are often really short snippets and I find it helpful to see more context.)

I then structured my code similarly to the code from csoni111.

  1. updateFem: I called a main updateFem() function once my document (the new tab) loaded.  That function contained the code to actually place my content on the page.
  2. getFemList: In order to be able to place content, I needed to get content.  I passed the function for placing content (my callback) to the next function which loaded the list of fems from my text file.
  3. randomFem: Once the list was loaded, I passed my callback (function to place content) onto the next function – the one that got a random name.
  4. getFemInfo: After I had a random name, I passed my callback over to the next function to get information from Wikipedia.
    1. callback: If I was able to successfully retrieve info from Wikipedia, I passed the info to the callback function to execute.
    2. getLocalFem –> callback: If I was unsuccessful (e.g. if I was offline when I opened a new tab), I retrieved a backup image from a local directory and passed this to the callback function to execute.

The callback was the very last function I wanted to execute.  I thought of it as an empty box with a bunch of gears inside that was getting handed from function to function.  Once all the other functions had executed, the final function handed all the necessary parts to my callback function and told the callback function to execute.

So that was the info retrieval side of things.  Read on to Part III to learn about how I displayed my content!

1 thought on “Coding Part II: Radfems”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s