How to call the reddit REST API using Node.js – Part IV

This is the last of a 4-part series that describes how to call and use the reddit REST API using Node.js.

In Part I, I talked about using curl to get your access token, which gets you permission to use the reddit REST API.

In Part II, I used that access token to call reddit’s search API. But I was still using curl to do this. I have a very large string output from the API, and I don’t know about you, but I’m not keen on using Linux command line tools to process strings. Since I like JavaScript, I decided to move to Node.js for completing my work.

In Part III, I built a Node.js script which gets my access token from reddit. That script does pretty much what I’d been doing in part I, but it does it using Node.js.

Today, I’m going to complete the work by adding a reddit search API call to my Node.js script, and then using JavaScript’s handy string processing functionality to display the information that interests me.

Recall that I’m pretending to be responsible for Starbucks public relations, and I want to find out what’s being said about Starbucks at reddit, in case I need to do damage control!!

Here’s my new Node.js method which uses reddit’s search API to look for new entries which mention “Starbucks”:

const searchReddit = function (d) {
	const options = {
		hostname: "oauth.reddit.com",
		port: 443,
		path: "/r/all/search?q=Starbucks&sort=new",
		method: "GET",
		headers: {
			"Authorization": "Bearer " + d.access_token,
			"User-Agent": "fullStackOasis NewPostsScraper"
		}
	}

	const req = https.request(options, (res) => {
		// console.log(`statusCode: ${res.statusCode}`)
		let chunks = [];
		res.on('data', (d) => {
			// d is a Buffer object.
			chunks.push(d);
		}).on('end', () => {
			let result = Buffer.concat(chunks);
			let tmpResult = result.toString();
			try {
				let parsedObj = JSON.parse(tmpResult);
				// Print the string if you want to debug or prettify.
				// console.log(tmpResult);
				processSelfText(parsedObj);
			} catch (err) {
				console.log("There was an error!");
				console.log(err.stack);
				// I got an error, TypeError: Invalid data, chunk must be a string or buffer, not object
				// Also I got this, when I'd pushed d.toString to chunks:
				// TypeError: "list" argument must be an Array of Buffer or Uint8Array instances
				process.stderr.write(err);
			}
		});
	})

	req.on('error', (error) => {
		process.stderr.write(error);
	})

	req.end();	
};

As before, you do not have to understand this code in detail to see what’s going on. The input to my searchReddit function has the access_token which I’d previously obtained in Part III. This new code uses that access token to call the reddit search API, doing a search for “Starbucks”.

Buried in that code above is a call to a function processSelfText. I need that because it’s not helpful to have a giant wall of text displayed to me! I need to process this blob of data, and have the script display only the interesting parts.

My function processSelfText grabs the blob of JSON which was returned from reddit’s search API, and loops through it for all the individual reddit threads. It prints out a substring of the thread that contains the mention of “Starbucks”, and also prints out the reddit URL in case I want to read the whole thread. I can quickly skim through the results to see if the thread looks potentially harmful to Starbucks. If it does, then I can go to reddit to respond.

Here’s the string processing code:

const processSelfText = function (obj) {
	if (obj.data && obj.data.children && obj.data.children.length) {
		obj.data.children.forEach(function (item, n) {
			// data is an Object. It may have selftext property
			if (item.data) {
				console.log("Item #" + n);
				if (!item.data.selftext) {
					console.log("Only found a url, no text:");
					console.log(item.data.url);
				} else {
					console.log("Found url and text:");
					console.log(item.data.url);
					showSurroundingText(item.data.selftext);
				}
			}
		});
	}
}

/**
 * Process the input string to 
 * @param {*} str 
 */
const showSurroundingText = function (str) {
	let maxchars = 150;
	// Have to do a lowercase search.
	let found = str.toLowerCase().indexOf("starbucks");
	if (found > -1) {
		// See https://davidwalsh.name/remove-multiple-new-lines
		str = str.replace(/[\r]+/g, " ");
		str = str.replace(/[\n]+/g, " ");
		// If first argument is too large, it's okay, just returns front of string.
		// If second argument is too small, also okay.
		var substring = str.substring(found - maxchars, found + maxchars);
		// Remove the new lines.
		console.log("..." + substring + "...");
	}
};

I run the script from the command line, like this: node reputation-checker.js. I am using Node.js version 8.3, and Ubuntu 16.04, but I think this script will work for most other operating systems and platforms. This is what the output of my script looks like:

Item #0
Found url and text:
https://www.reddit.com/r/aznidentity/comments/cybh97/we_are_not_honorary_white_people_and_do_not/
 …tention span which many folks nowadays unfortunately do not.  I’ll order my morning coffee at an Asian-owned establishment rather than an overpriced Starbucks where it ends up tasting burnt anyway.  I could not care less for Mexican food or a Westerners version of Chinese food. Authentic Asian cuisi…
 Item #1
 Found url and text:
 https://www.reddit.com/r/exmormon/comments/cybd66/mom_why_dont_you_get_a_blessing_me_because_they/
 … of my skin alternates between burning and itching. Sometimes, I get both at once. Which is what happened yesterday. While I was waiting in line at Starbucks, talking to my mom on the phone.  My mom knows I left the church. And she knew I was at Starbucks. But none of that should matter. She’s livin…
 Item #2
 …

I could do more to refine this code – normally, I’d refactor it, and write some tests, and do more error handling, maybe automate it to send me an email periodically… but this is good enough for demonstration purposes. Here’s a link to the entire script, if you want to download it and mess around with it.

I hope you enjoyed this tutorial! Please feel free to use the “subscribe” form if you’d like to keep posted on updates to the “Full Stack Oasis” blog. I only post about once a week.

If you found this interesting, click the subscribe button below! I write a new post about once a week.

Leave a Reply

Your email address will not be published. Required fields are marked *