Monthly Archives: August 2019

How to call the reddit REST API using Node.js – Part II

In my previous post, I showed how easy it is to authenticate using the reddit API. Authenticating leaves me with an “access token”, which is just a JSON string which looks like this:

{"access_token": "261295382351-FBXDPTpUam35NR_UTJSXnjl5Pmd", "token_type": "bearer", "expires_in": 3600, "scope": "*"}

The access token gives me permission to use the reddit API. Now I can actually write an application that consumes the reddit API.

In order to motivate this demo, suppose you’re in charge of reputation management at Starbucks, and you want to be notified if anyone says anything about Starbucks at reddit. Fortunately, there’s a handy search API which can be used to search for the word Starbucks.

Here’s how you can use it to search for any mention of “Starbucks” at reddit:

curl -H "Authorization: bearer 261295382351-FBXDPTpUam35NR_UTJSXnjl5Pmd" --user-agent 'my test bot 1.0' -A "my app identifier" https://oauth.reddit.com/r/all/search?q=Starbucks

This returns a list of 25 results. If you check the search API that I linked to above, you’ll find that the default number of results (“limit”) is 25. The documentation doesn’t mention how results are sorted, but it looks like they are sorted by “relevance”, by default. We’d rather get them by “new”, to get the most recent posts. So let’s add that to the query string.

curl -H "Authorization: bearer 261295382351-FBXDPTpUam35NR_UTJSXnjl5Pmd" --user-agent 'my test bot 1.0' -A "my app identifier" https://oauth.reddit.com/r/all/search?q=Starbucks&sort=new

Now, your output is looking pretty hairy – a giant string of JSON text. You can take a look at this output in an online JSON “prettifier”, if you want to get a better idea of what’s being returned. This will be easier if your output is in a file. You can redirect the output of curl to a file. Just make sure that you put your URL in quotes, because the ampersand is interpreted by the shell to mean “start this command in the background”, which is not what you want. Do this:

curl -H "Authorization: bearer 261295382351-FBXDPTpUam35NR_UTJSXnjl5Pmd" --user-agent 'my test bot 1.0' -A "my app identifier" "https://oauth.reddit.com/r/all/search?q=Starbucks&sort=new" > data.json

Notice that so far, all I’ve done is use curl, a command line app, to call the reddit API! Now that I have a pretty good idea of how I’m going to be using the reddit API, I will start writing my Node.js application. I’ll dive into that next week.

If you found this interesting, click the subscribe button below! I write a new post about once a week.

How to call the reddit REST API using Node.js – Part I

One common programming task is to call a REST API. So it’s a good thing to know how to do this! In this demo, I’ll write a short Node.js program which “consumes” the reddit REST API. I’ll be doing this using a Linux environment (Ubuntu 16.04).

First, I had to sign up for a developer account at reddit. I was already a member of reddit, so I just used my own information to create the “app”. The reddit API access page is the place to go for signing up. There’s a link at the bottom which has the text “Read the full API terms and sign up for usage” – click that, and follow the instructions.

Next, go to your “Developed applications” page. Your “id” is found at the top left, a random string under the words “personal use script”. Click the “edit” link, below, and you’ll find your “secret”, another long string which should not be shared!

Using the reddit API requires authentication. Reddit needs to be able to know who is using their API, and to place limits on it, in case of abuse. So I need to figure out how to do this first.

Now that I’ve got an “id” and a “secret”, I can follow the instructions for authentication using OAuth2.

Let’s test the authentication process using curl, a command line tool which lets you “get” a web page from the command line.

curl -X POST -d 'grant_type=password&username=my_user_name&password=MyExcellentPassword' --user 'my_reddit_id:my_reddit_secret' https://www.reddit.com/api/v1/access_token

Copying and pasting that won’t work for you! Make sure to replace the strings my_user_name and MyExcellentPassword shown above with your own reddit sign-in information, and use your own id and secret as referenced above, when using curl.

After I did this, I saw the following output:

{"message": "Too Many Requests", "error": 429}

Oops. It turns out you need to set a user agent header to avoid this. Curl has a specific option, --user-agent, to let you do this. I just used a fake user-agent for my demo purposes: “my test bot 1.0”.

curl -X POST -d 'grant_type=password&username=my_user_name&password=MyExcellentPassword' --user 'my_reddit_id:my_reddit_secret' --user-agent 'my test bot 1.0' https://www.reddit.com/api/v1/access_token

Success! My output from curl looks like this:

{"access_token": "261295382351-FBXDPTpUam35NR_UTJSXnjl5Pmd", "token_type": "bearer", "expires_in": 3600, "scope": "*"}

This token will expire shortly… but that doesn’t matter. Now that I’ve got the authentication part working, I can work on the application, itself. I’ll cover that in my next blog post.

If you found this interesting, click the subscribe button below! I write a new post about once a week.

Freelancer PSA : Do not tell the client you are going on vacation

I think most freelancers originally worked as an employee at some time in their lives. As an employee, it becomes a habit (by necessity) to tell your employer that you are planning to take a vacation, or you need sick time, or family leave.

Once you become a freelancer (contractor), please keep this in mind: your client is not your “employer”. Any arrangement between the two of you requires your consent, not just theirs. It is “just my opinion”, but you should not feel a need to justify your schedule to a client, unless you’ve signed a contract which says you will do some specified piece of work for some specified hours during some specified time period.

Here is one example: A client hires you to complete a project in July of this year. Once the work is done, your relationship with them is officially finished. Don’t feel you need to tell them “I have a vacation scheduled in September”, just in case they might need you again at that time. That is none of their business! Once you start doing things like that, it plants a seed in the client’s mind that you will be available unless you warn them otherwise!

However, if you detect some expectation that they might be planning or assuming that you will be available for future work, you may want to caution them that you will not be available going forward without an agreement in place. For example, suppose you finish your batch of work, and the client emails you something like “We may need help in creating more content in the fall of this year.” You can send an email back saying “I am not sure of my availability at this time. Please send me an email with further details when you are more sure of the schedule and the amount of work needed. If I am available, I would be happy to help you.” You might want to mention a retainer agreement at this time.

Suppose the client thinks that they might need you for additional work in September. In that case, they should make an agreement with you in advance that you will be there for them – this is a “retainer”. So, they pay you $$ in advance so that you carve out time which is reserved for them – and the agreement specifies that you keep the money whether or not they use your services. Remember, they are reserving your time with the payment. You could have been working for someone else during that time period, if you had not reserved it for them. You should not be losing money because you set aside time to work for them but they were too disorganized to make use of it.

So, if you finished your work in July, and the client signed off on it, they have no right to send you an email at the end of August which says “We need you for work in September.” It’s actually kind of rude to do this. They’d be making an assumption that you are twiddling your thumbs, waiting for new work from them, which is generally not the case for freelancers.

Two of the things you need to do as a freelancer is to “manage client expectations“, and “set boundaries“. Boundary-setting means the client doesn’t get to know if you are on vacation, giving a work seminar in a remote location, or anything else. It is not their business. (I am speaking from a USA-centric point of view, fwiw).

Managing client expectations, in this respect, just means that the client sometimes needs to be reminded that you are not an employee. Managers who are only used to working with employees sometimes forget that contractors are not machines that are instantly available. Sometimes, they are used to pressuring contractors to behave like employees, with the implication that they will not give them future work unless they are always on call, respond to emails instantly, etc.

It is up to you how you respond to such pressures, of course. In the long term, my belief is that you are better served by pushing back when clients start to act like your “boss” – because they are not! There’s a risk that this client might “fire” you, and get someone else to bully (good riddance!). Hopefully, though, they will learn to be respectful of your time.

If you found this interesting, click the subscribe button below! I write a new post about once a week.

How to create a keystore file for Android development using the command line environment

TL;DR: Run keytool -genkey -alias MyReleaseKeyAlias1 -keystore mykeystore.pfx -storetype PKCS12 -keyalg RSA -validity 1095 -keysize 2048 and respond to the prompts to generate a keystore file which is good for 1095 days (3 years).

Even the long answer is pretty straightforward 🙂 I am using a Linux environment, Ubuntu 18.04.

You may need to install the JDK. Make sure that JDK version 8 is installed (if the JDK is already installed, this command will not do any harm, so you need not be afraid to run it):

sudo apt-get install openjdk-8-jdk

Next, create a directory for your keystore, and go into it. For example,

mkdir ~/AndroidKeystores
cd ~/AndroidKeystores/

(The tilde ~/ is a shorthand for your home directory, for example /home/fullstackdev.)

Finally, generate the keystore by using keytool

keytool -genkey -v -keystore my.keystore -alias MyReleaseKeyAlias1 -keyalg RSA -keysize 2048 -validity 1095

You will have to answer a few, fairly straightforward, questions. In my example, below, I just made up some silly answers; you should use your own information instead. But you can also make up anything – as long as you are just using the keystore for development work, it doesn’t matter what you use.

keytool -genkey -v -keystore my.keystore -alias MyReleaseKeyAlias1 -keyalg RSA -keysize 2048 -validity 1095
Enter keystore password:  
Re-enter new password: 
What is your first and last name?
  [Unknown]:  Full Stack Dev
What is the name of your organizational unit?
  [Unknown]:  Full Stack Oasis
What is the name of your organization?
  [Unknown]:  
What is the name of your City or Locality?
  [Unknown]:  Whitehorse Ledge
What is the name of your State or Province?
  [Unknown]:  New Hampshire
What is the two-letter country code for this unit?
  [Unknown]:  US
Is CN=Full Stack Dev, OU=Full Stack Oasis, O=Unknown, L=Whitehorse Ledge, ST=New Hampshire, C=US correct?
    [no]:  yes
Generating 2,048 bit RSA key pair and self-signed certificate (SHA256withRSA) with a validity of 1,095 days
	for: CN=Full Stack Dev, OU=Full Stack Oasis, O=Unknown, L=Whitehorse Ledge, ST=New Hampshire, C=US
Enter key password for <MyReleaseKeyAlias1>
	(RETURN if same as keystore password):  
Re-enter new password: 
[Storing my.keystore]

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore my.keystore -destkeystore my.keystore -deststoretype pkcs12".

Now, notice that the output tells me that JKS is proprietary. So let’s use the recommended solution:

keytool -importkeystore -srckeystore my.keystore -destkeystore my.keystore -deststoretype pkcs12

Here’s the output:

Enter source keystore password:  
Entry for alias myreleasekeyalias1 successfully imported.
Import command completed:  1 entries successfully imported, 0 entries failed or cancelled

Warning:
Migrated "my.keystore" to Non JKS/JCEKS. The JKS keystore is backed up as "my.keystore.old".

Since I migrated the key to PKCS12, I might as well have created the key that way, too. Here’s how that’s done:

keytool -genkey -alias MyReleaseKeyAlias1 -keystore mykeystore.pfx -storetype PKCS12 -keyalg RSA -validity 1095 -keysize 2048
Enter keystore password:  
Re-enter new password: 
What is your first and last name?
  [Unknown]:  Full Stack Dev
What is the name of your organizational unit?
  [Unknown]:  Full Stack Oasis
What is the name of your organization?
  [Unknown]:  
What is the name of your City or Locality?
  [Unknown]:  Whitehorse Ledge
What is the name of your State or Province?
  [Unknown]:  NH
What is the two-letter country code for this unit?
  [Unknown]:  US
Is CN=Full Stack Dev, OU=Full Stack Oasis, O=Unknown, L=Whitehorse Ledge, ST=NH, C=US correct?
    [no]:  yes

I did the same thing this time as I did before. But now I’ve generated a PKCS12 keystore, and got no complaints from keytool. Now I can use the output keystore file, mykeystore.pfx, for signing Android apps. Task complete!

Here’s one last hint. If you build your Android APK using gradlew, make sure that you delete your release build APK prior to rebuilding, if the only thing you’ve changed is the keystore file. Gradle will only build your APK if it sees a change in the source, and your keystore file is not in your source code. So Gradle won’t rebuild when you’ve only changed the keystore, unless the APK is gone. You may be fooled into thinking that the APK was built, until you notice that the timestamp on your APK is old. This happened to me 🙂

If you found this interesting, click the subscribe button below! I write a new post about once a week.