My Google, Shanghai: Explained
I've talked a little about being in China, but I haven't said much about why. Up until only recently my duties at Google were unclear, but now I understand my purpose: Christophe dragged me over here to contribute to Hadoop, an open-source MapReduce implementation.
Hadoop is essentially a tool used by software engineers to write programs that use large amounts of computers to process vast amounts of data. Cloud computing is the new buzz word, but Google revolutionized large-scale computing, or distributed-computing, many years ago. Historically, lots of data (like that of the internet) was analyzed by large, expensive computers. In fact, historically, lots of data just flat out wasn't analyzed. Now, in the wake of MapReduce, Hadoop puts hundreds or even thousands of commodity computers to work to analyze data. Cloud computing is one of the reasons why Google is the best search engine, and industries all over are benefiting from the cloud. Cancer researchers are able to more efficiently understand their data. Astronomers can crunch their images much faster. Hadoop allows any company to effectively understand large amounts of data.
It's not yet clear exactly how I'll be contributing to Hadoop; those details should surface soon. I admit that Hadoop is my first open-source project, and I'm very, very excited to be contributing to a field that is growing so rapidly. More updates to come!
Bonus story: after slaving away for four days, I finally have Hadoop's trunk build running on a multi-node cluster. Boom shakalaka!
A Second Look at Software Engineering a Startup
I wrote a post a while back about software engineering a startup, and Clint's comment made me realize that my post was poorly thought out and just plain bad. I argued that the quality of code you write should be related to the type of financial situation you're in. For example, if you're a funded startup, then you should be writing good, maintainable code. However, if you're not funded or if you're doing a personal project, then you should code quickly to get features out the door. I want to revise my argument.
I've now worked on two pretty large personal projects: Cellarspot and Best Seattle Bars. Cellarspot was developed in JSP, and the code we wrote was totally maintainable and robust. On the other hand, Best Seattle Bars was developed in PHP, and the code I wrote was all inline. The following analysis may be obvious, but I'll make it anyway:
Robust, Maintainable Code
Advantages: Maintainable, easily refactorable, and expandable
Disadvantages: Generally requires more work, thought, and usually a more advanced development environment
Inline Code
Advantages: Requires less work, less thought, and a much less sophisticated development enrivonment
Disadvantages: Not maintainable, not easily refactorable, and not expandable
So when should you write fast, inline code, and when should you write good, robust code? Inline code is great when you want to prototype something. It allows you to quickly launch a set of features so you can evaluate your idea. Robust, maintainable code is basically great for all other cases. Now let's examine three cases:
You write inline code and your idea blows up
Your idea takes flight and before you know it you have tons of users, tons of peers programming with you, and your users want more features and better reliability. Chances are good that you'll have to recode your entire web app, and that's going to be a HUGE upfront cost that perhaps is more costly than writing good code to begin with.
You write good, maintainable, robust code, and your idea flops
First, let me just say that I feel your pain. One might think that this case is a total waste of time, but I'll argue the contrary. I learned an insane amount by going through the process of writing good code, and it would be a good practice for me to do it again. The value one gains by going through this process is huge - it can help you get jobs (it did for me at least), and it keeps you sharp.
You use Ruby on Rails or some similar framework
Frameworks such as Ruby on Rails and Django let you write good, maintainable code very, very quickly. They let you avoid the above two cases completely, because you won't waste your time on a flopped idea and you won't find yourself scrambling for dear life if your idea blows up.
My conslusion is the following: write good, maintainable code, and choose a framework that makes it easy for you to do so. You'll avoid the upfront costs of writing good code, and you'll avoid the posiblity of large refactoring down the road.
Google’s Earth Day
It appears as though Google has beaten me to the punch again. A few months back I wanted to create a user-driven, Q&A-style search engine. Two days after I got my idea and started coding I learned that Google had done this in Google Labs and that Mahalo was already doing this. Dang.
Another recent idea I had was to create an index of non-profit organizations in hopes of allowing people to become more aware of the organizations that are out there. Google just announced that as well. I suppose it's better Google than me - they have huge traffic and will most likely generate a lot of donations. If my non-profit index would follow suit with all of my other websites such as Helpd, which has somewhat of a similar goal, then the non-profits would get little to no donations. Go take a look at what Google is doing, and donate if you want. It'll probably help out 
. I donated to Acterra.
More Best Seattle Bars Improvements
In my quest to learn more about JavaScript, I launched another iteration of Best Seattle Bars that includes:
- Searching (searches the name, the description, and all reviews)
- Adding of new bars by the user
- More user profile fields (name, email, phone)
- Framework for SMS and email announcements
- Promotion integration (BSB is participating in the Fremont Troll Stroll to benefit Cystic Fibrosis)
Next up is probably photo adding and possibly a few others. Any suggestions? Hope you enjoy!
Best Seattle Bars Improvements
Best Seattle Bars is so far the most successful website that I've ever launched. The guy with the idea, Kyle Trew, loves to hit the bars, and he promotes the site almost every chance he gets. Almost half of our traffic is direct traffic, which implies that his promotions are definitely working. Anyway, I launched a new version of the site just a few minutes ago. I've been working on this new version for about two weeks, and it's worlds better than the previous version. Here's a feature list:
- Improved map user experience
- User registration
- 5-star ratings
- Improved bar landing pages
- Driving directions
I've already started working on the next version, which should again be a huge improvement.
Go take a look and let me know if you have any suggestions. Enjoy!
Networks Project: Radical
My friend, Robert, and I finished our big networks project about 2 weeks ago, and I just wanted to share the project. We were given Nokia N800s and told to write a P2P file sharing application in C. Awesome.
Phase 1: The Basics
We follow the specified protocol and write a program that lets N800s request lists of files and also request files from their neighbors. Neighbors are discovered by UDP broadcast packets that are sent every few seconds. All other communication – file list and file requests – are done over TCP. I'm leaving a lot of detail out such as port selection and protocol details, but the project was relatively simple and fun – lots of buffer/network code.
Phase 2: Routing
We build off of phase 1 and implement a link state routing mechanism. We also use Dijkstra's algorithm to compute the shortest path between all nodes. We build a ping and ping reply feature to test our routing. Link state packets and ping packets are sent via UDP. This part of the project was long, hard, and awesome - tons of fun data structure/algorithm code.
Phase 3: Enhancements to Phase 1
We build a file search and request mechanism so N800s can ask their neighbors for a particular file instead of first viewing a list and then downloading. Neighbors queue requests that they can't fulfill and send these queued requests to their neighbors. Files are downloaded automatically when a request is fulfilled. I can't remember what network transport layer is used, but I'm pretty sure file requests and downloads were both sent over TCP. Again, a really fun phase.
Conclusions
This project was awesome. Robert and I probably spent at least 100 hours each in the labs, and we enjoyed most of it. The biggest pains were cross-compiling and running on the N800s and odd GCC behavior. If you're a UW CSE student and plan to take networks (CSE 461), then try and take it with Arvind Krishnamurthy. Robert and I both have a few complaints about the class, but the project was tons of fun. We ended up writing around 4,000 lines of C code from scratch, including a generic hash map, list, set, and many-to-many table. Great project. I wanted to post the source code, but then I realized that I probably shouldn't 
.
Viral Sites, the Blogosphere, and SEO
I wanted to say more about viral sites. I think there are two really cool features that can be implemented to improve your SEO and also utilize the Blogosphere more.
Widgets - get people linking to you
Offering widgets makes it very easy to get people to link to you. For example, I could create a widget for Best Seattle Bars that allowed people to drop some code on their site and display a list of the best Seattle bars. I could also create a widget for Best Seattle Bars that allowed people to drop some code on their site to display a search box. The two best advantages of using widgets are 1) you get people to link to your site, which improves SEO and also increases referrals, and 2) you get to control the way that people link to your site. Here are two examples of widgets that Trulia, a Redfin competitor, offers:
Pretty nifty, huh?
Trackbacks - encourage bloggers to link to you
Bloggers love to blog. If you have rich content on your site such as homes for sale, restaurant reviews, or travel tips, then chances are good that bloggers are already blogging about you and linking to home, restaurant, and travel pages. Reward them and maybe they'll blog about you more. Setup a mechanism on your site to capture trackbacks sent by blogs, and put a link on the page that's being linked to, linking back to the blog post. On Best Seattle Bars, an example would be to put a link to each blog post that links to a particular bar page at the bottom of that bar page. You would probably want the link to be nofollow, but bloggers would still get referrals from these links.
The largest factor of PageRank is link analysis, so the more people that link to you, the better you'll perform in search engines. The better you perform in search engines, the more people you'll get coming to your site.
helpd.org, Reworked
LN, my former dorm neighbor and good friend, recommended that I rework helpd.org. I reworded the site to motivate people to post ways that other people have helped. I really want the site to just be an anonymous, free-form way for people to share stories about being helped or helping out, and LN helped me realize that. I also added a RSS feed so you can subscribe to updates. I'm open to suggestions on how to improve the site; write a comment.
Share some stories! Thanks.
Introducing Best Seattle Bars
Today is the official launch day of Best Seattle Bars! I made the site for a friend of mine, and I had a blast doing it. Features include:
- Interactive map of the best Seattle bars
- Popular neighborhoods
- Basic bar information (name, address, phone, and website)
- User reviews and a wiki-style description for each bar
- An RSS feed for review and description updates - check it out
Given that Kyle's domain is pretty unbelievable, I wanted to make the entire site as SEO driven as possible. I have pretty URLs, and keyworded titles and headers. Kyle and I are hoping to get tons of organic search traffic, and we're hoping that the RSS feed gets people coming back for more.
I wrote the site in PHP with a MySQL backend. Everything is inline, and I'm really glad I made the decision to not write good code. I've wasted too much time writing good code in the past. Eric did the layout and design - I think it took him 30 minutes.
Kyle has some pretty awesome ideas with this, so stay tuned for more updates. Enjoy!