Node.js experiment part 2: Research, review, and learning

I want to do this experiment right, so rather than jump right in I’ve decided to do a lot of research and planning before hand.  I’ve also decided to review JavaScript and learn Node.js “correctly” by reading introductions and tutorials.

But two things before I begin:

  1. I need a name for the project!  It’s bothering me that I don’t have one, and I don’t want to go through the bother of going with a temporary name and renaming everything afterward.  Any ideas?  It needs to describe the “resource blog” idea, but also maybe be cute and easy to remember.  These are some random thoughts I’ve jotted down:
    • bacon links?
    • piggy links?
    • library?
    • repository?
    • reference?
    • collection?
    • anthology?
    • compilation?
  2. I want to be organized, so I’ve started a notebook for this project.  I write notes on introductions/tutorials in it, and also just jot down any thoughts I have.  I like writing things down on paper rather than just typing them, because it helps force me to think through it – when I type, sometimes I type so fast I can’t even think fast enough to keep up.  I also discovered Trello, this awesome “collaboration tool” that’s perfect for organizing a project, especially projects with multiple people.  I found it when I was looking through code52 about their most recent project.  Here is their Trello board for their project Samurai.  It’s awesome – there are lists of different types of tasks, and you put cards in the lists, which represent tasks.  The cards can have notes, comments, and checklists, and can be labeled with categories and even have people assigned to them!  Here is my Trello board for my experiment.  I’ve just started putting it together so I’ll definitely be expanding it.  Comments on the cards are welcome :)  Also I feel silly assigning myself to everything, but when I finish things it makes me feel like I’ve accomplished a lot!

Here’s a table of contents again:

Research
Review
Learning

Research

First I expanded my feature list so I could figure out what I would need to do.  Here is what I have in my notebook:

  • Log in to post
    • Only me for now
    • Goes to admin center – like the WordPress admin page
      • Add/edit/delete resources
      • Save versions eventually?
      • Save drafts
      • Moderate comments
      • For posts:
        • Create/select category (can only belong to a single category)
        • Add tags
        • Title
        • URL
        • Note
        • Rating
    • In the future, expanding for more users
      • Admin can manage users
      • Manage user profile – name, email, picture, URL, bio, etc.
      • Add/edit/delete own posts
      • Versions?
      • Drafts
      • Same as above for posts
      • Permissions model – everyone sees same functionality but can only act based on permissions
  • View resources
    • Summary view and full view
      • Summary view – multiple summary views on a page, “more” link will show full view
        • Title
        • URL
        • Snippet of note
        • Rating
        • Category
        • Tags
        • Number of comments
        • Votes with ability to vote (like Reddit)
      • Full view – each resource on a single page
        • Title
        • URL
        • Note
        • Rating
        • Category
        • Tags
        • Votes with ability to vote
        • All comments
          • Name
          • URL/email
          • Comment
        • Add comment
          • Name
          • URL/email
          • Comment
          • Recaptcha
    • Default view
      • Display resources by time, most recent first
      • Show each resource in summary view
    • Category view
      • All on one page
        • Alphabetical categories
        • Show in category tree
        • Show minimized by default, can maximize to see summary view
      • By category
        • Same format as all on one page except each page is for each outmost category (subcategories don’t get own page)
        • Can sort by date, rating, votes
    • By tags (user search by tag)
      • Show summary view of results, by date as default
      • Can sort by date, rating, votes
      • Minimized by default, maximize to see summary view
    • By highest rated (my rating)
      • Same format as tags view
    • By highest voted (by visitors)
      • Same format as tags view
  • Index page
    • Default view of resources
    • Tag cloud
    • Short list of highest rated with “more” link
    • Short list of highest voted with “more” link
    • Link to page with different category views
    • Search bar for tags
    • About link
    • Contact link

Now that I’ve typed it out, I realized that there may be too many different views of a single resource – full view, summary view, and minimized view.  I need to think more on that.  I figured that showing the full view of every resource would be too much information if it wasn’t the index page so I think there should be at least two views, but it may not be necessary for both a summary view and a minimized view.  I may get rid of the minimized view, otherwise someone would have to first expand the minimized view, and then click “more” on the summary view to see the full view.

I’ve also come up with a list of tools/frameworks that I think I’ll need for the project.  I did quite a bit of research on the best tool for each function, and I think this is the finalized list.  They’re in no particular order under each category:

  • The general code – server and client side
    • Node.js – obviously
    • Express – web development framework
    • Jade – template engine
    • Stylus – CSS pre-processor
    • nodeQuery – DOM manipulation framework with JQuery methods
    • everyauth – authentication and authorization
    • Async – utility module for asynchronous code
    • node-UUID – generate UUIDs
    • node-recaptcha – renders and verifies Recaptcha captchas
  • Database
    • MongoDB – document-oriented NoSQL database system
    • Mongoose – MongoDB object modeling tool
  • Testing and debugging
  • Deployment
    • Forever – tool to ensure a script runs forever
  • Logging
    • winston – multi-transport asynchronous logging library
  • Documentation
    • NDoc – documentation generator

I’ve always had this really weird attitude towards using external tools.  On one hand, I feel like doing everything myself is “better” in terms of me being a programmer and not being lazy.  But at the same time I guess it’s not lazy, it’s being resourceful and using what’s already out there.  I never know how I feel about it… I still get conflicted about it all the time.  But I’ve got this huge list now, so I’ll have to deal with it!

There are still some things I need to consider for the project, and need to do research on – I won’t fully go into them here but they’re on the Node.js Experiment Trello board.

Review

Since all my JavaScript is self-taught, and JavaScript is kind of important to Node.js, I decided to fill myself in by following a introduction tutorial.  I also want to learn to use Node.js the “right” way, so I am following an introduction tutorial for that too.  I decided on Felix’s Node.js Beginners Guide, which was suggested by a lot of people when I was researching best Node.js tutorials.  And as luck had it, that tutorial suggested Eloquent JavaScript as a JavaScript tutorial, and it had good reviews when I researched it, so I’ve decided to follow that for my JavaScript review.

I’m up to chapter 9 in Eloquent JavaScript and it’s been great for me so far, as someone who already has programming background in other languages and wants to just pick up the little nuances about JavaScript (it still throws me off that you don’t have to tell a variable what kind of variable it is.  And null vs undefined?  == vs. ===?  Closures?!  Prototypes?!?!)  However I think that as an introduction for someone who has never programmed before, it’s kind of a mess.  It doesn’t explain things very clearly and kind of brushes over some programming concepts as “magic”.  I had to do some outside research about closures and prototypes, and I still don’t fully understand them.  They also try to shove too much down your throat – they had an entire chapter that was based around the concept of graphs but never explained graphs and just sort of threw code at you.  I love the in-browser console though; it’s really easy to follow along with their code samples and edit it myself.  Try out the in-browser console – it’s awesome.  Overall I think it was a good choice.  Still need the finish taking notes on the last few chapters though.

I’m also playing around with Codecademy to keep filling in any knowledge gaps I have.  It’s neat, I like the tutorial format, with the skeleton code in the editor and the console to run it.

After that I’m going to take notes on Felix’s Node.js Beginners Guide.  Then I’ll see what other knowledge I may be lacking and research some more good tutorials.

Learning

I guess I kind of already covered this with Felix’s Node.js Beginners Guide.  But I’m also going to need to become familiar with all of the tools and frameworks I’ve chosen to use in the experiment, so I think that I will have to do some reading on each of those and code through some small sample projects for each of them, before I try to use them all together.  I still need to come up with a plan for that – not sure what order I’ll be going in, as the list of tools and frameworks above are in random order.  I wonder if it’s better to familiarize myself with testing tools first so that I can use them while I learn the others?

Anyway, it’s late now, so I’m going to bed.  Goodnight!

Node.js experiment part 1: Defining my experiment

I’ve decided that my technical project will revolve around Node.js.  And now I will start my long convoluted story about how I came up with my project…  I guess I will provide a table of contents again since I tend to ramble a lot :)

The back story
The idea
The goals
Future considerations
The plan

(I just want to say here that I accidentally deleted a whole bunch of stuff I wrote and WordPress didn’t auto-save a draft before it happened so now I am an extra sad panda.)

The back story

At work, I was working with part of our team in China, and right before the winter vacation they gave us the code to a web socket server in Node.  I’ve always been interested in web technologies – I taught myself HTML from a book in fourth grade and self-taught whatever PHP I needed in middle school for my embarrassing (and luckily now-nonexistent) EatenCheez.com – but I’ve never had any formal training.  After I moved to Virginia in high school, I stopped playing around with HTML and PHP in my free time, and since I focused on embedded systems in college, I don’t really have any idea what Node was.  I was curious, so I looked into it, and it’s so fascinating!  A web server written in 6 lines of code?  Awesome!

(Side note Easter egg – I found a snapshot of EatenCheez.com from 2003.  I don’t know what happened to the formatting but I thought it was a funny reminder of way back when.  Cliques, rings and clubs?  Guestbooks?!  At least this version of EatenCheez.com didn’t have a splash page with the obligatory hit counter!)

I started looking into Node, reading about what it’s best used for and following some cool tutorials.  I even installed Ubuntu on my Windows desktop just to be able to play with Node easier.  There are some really cool tutorials and projects out there – such as this Scrabble MMO written in 48 hours, this Twitter clone, or this awesome phone-controlled multi-player browser game written by an intern (try it out, it’s neat and cleaned up my rusty French) – but following someone else’s tutorial isn’t the same as creating your own project.  So I kept looking into Node to see if I could get any inspiration.

The thing that ended up really inspiring me ended up being… my laziness.  When I’m bored, I will lounge around on the couch or on the bed browsing the internet on my phone.  However, phone screens are tiny!  So when I find an interesting link, I will email it to myself to read later.  I like how mailing a link to myself from the iPhone will put the title of the page in the subject and the URL in the body of the email, because the title is obviously much more descriptive than the URL.  However, this means I end up with an inbox looking like this (I actually had 20 sitting in my inbox all day but I started moving them before I remembered to take a screenshot):

(I’ve been having insomnia for the last week, only getting 3-4 hours of sleep a day… hence the very early morning emails to myself)

I wanted to clean up my inbox, but I didn’t want to just have a big list of URLs and no descriptions, so I decided to just move things to a text document:

Now my email inbox was clean, but I had a list of titles and URLs on my desktop that I couldn’t access from anywhere else.  That is the same reason I don’t like to use browser bookmarks – I’ll put them there, then not be able to access them when I’m, say, at work and have some free time to read.  I started syncing my Chrome settings recently and they sync bookmarks, but I also use different browsers when I’m working, so I’d have random bookmarks saved across multiple browsers on multiple machines.  I tend to just avoid browser bookmarks in general.

On Saturday I was in the middle of this tutorial for a blogging system in Node when it hit me – I should make my own personal resource blog!

The idea

“Personal resource blog” doesn’t explain much.  I was formulating this idea while I was (yet again) lounging around on my phone, so my notes ended up like this:

Basically I want to make a “blog” where each post is a useful resource I found, so I guess a library of bookmarks.  Here is a cleaned up, better explained, and expanded version of this list of features I made.

  • Log in – Since it is a personal blog, only I can log in and post
  • Create category (can be nested) – Each resource is listed under a category, and there can be categories within categories
  • Select category (tree view) – If I already have the category I need, just select it
  • Input title, link, comments, tags – Each resource consists of a title, the link to the page, my comments about how useful I found it, and some tags.  I’ve also been thinking that maybe I could rate it.
  • Submit – submit the resource, obviously
  • Categories displayed in multiple ways (all on one page, each category level on one page, each resource on one page) – Now I’m getting into what the visitors see.  They could view all and see the resources listed under their categories, or they could select a category and view only those resources on a page.  I believe that in the last part I was referring to each resource having its own permanent link, so perhaps in the two previous views not all of the information would be shown, and the permanent page would be the full view.
  • Search by tags – Users can search by the tags on each resource
  • Tag cloud – Always nice to see a tag cloud
  • Can add comments to each resource – User can comment on resources about what they think
  • Can vote up – Or down too I guess.  Like stackoverflow or reddit
  • Need to link to database – I guess this wasn’t really a feature, I was writing a note to myself that I need to look into different databases
  • Sort by my score – Users can see what I liked best
  • Sort by date – See when I added resources
  • Sort by votes – See what everyone thinks is the best

Those are my ideas as of now.  I think that I could definitely expand it – for instance, if it actually works and I don’t code up a catastrophe maybe I could make it so users can make accounts to submit resources so it wouldn’t just be a personal library any more.  I don’t know, I’m mostly using this as an experiment to learn, so I don’t even know if people would be interested or find something like this useful.

The goals

The main goal is obviously to learn to use Node.  However as I was looking into it there are also many other useful tools to learn and utilize in the process.  Here are the ones I’ve discovered so far that I would have to learn, use, and become more familiar with (on top of polishing my very rough Javascript):

  • Node.js – server-side Javascript
  • Express – Node web framework
  • Jade – Node HTML template engine
  • Stylus – Node CSS template engine
  • JQuery – popular Javascript library
  • MongoDB or CouchDB (or both) – NoSQL database systems

And of course the end goal is to create my own CMS.

Future considerations

There are quite a few things I need to think some more about, which will come in future posts.

Full definition of features

I need to sit down and write down exactly what I want before I do any coding.  I think the list I have above is a good start, but I need to clean it up some more.

github and open source

I’ve never used github before, but I think that I should make use of it.  Of course this means that since I am cheap and will be using a free account, my project will need to be open source.  I want to learn more about the idea of open source and different open source licenses.  I’ve already done a bit of reading, and I think one thing I need to do is look at the different libraries and frameworks I’m using to see what open source licenses they use, then pick one for my project.  Or is it even that important?  I have no idea, I’ve never dealt with any of this before…

Hosting the project

BaconFriedRice is on a shared hosting plan on A Small Orange, so I probably shouldn’t put my app there since it’d be persistent.  I’ve been looking into different places I can host a Node app, but I realized there’s quite a bit to think about.  I want to start off writing it on my machine, then deploy it on some free hosting as I test it.  The only problem is that the databases for free hosting are either non-existent or very small, and there are also limited options – for instance, I was looking at Cloudnode, but they only use CouchDB (and you only get 25MB), while it seems like more people use MongoDB.  But then, if all goes well and I actually write this thing, maybe I would just pay for hosting in the future. It’s hard to say right now… but I’ve also been looking into Nodester, dotCloud, Webbynode, Joyent Cloud Services, and Cure.  Also database hosting at MongoHQ and Cloudant.

Picking a database

Like I mentioned, the two options I’ve been looking at are MongoDB and CouchDB.  They seem to be the most popular for Node.  However it really depends on where I end up hosting.  I actually think that after I finish the blog tutorial with MongoDB, I’ll do this blog tutorial with CouchDB and see which I like using better.  That’s probably a better way to decide after all!

Picking a name

This is my top priority right now.  I need a name for my project!

The plan

  1. Finish the MongoDB blog tutorial
  2. Do the CouchDB blog tutorial
  3. Pick a name!
  4. Flush out the features and put them in order to add one by one
  5. Pick a database
  6. Pick a free hosting to start with
  7. Start coding!
  8. Utilize github
  9. Keep coding
  10. Figure out the whole open source thing
  11. Don’t stop coding
  12. Sleep a bit
  13. Code some more
  14. If it actually works… Move it to paid hosting with more space!
  15. Celebrate

Sorry for the very long read :) Now it’s time for bed!