Shady Acres

Choosing your battles

One of the top items on the Hacker News site today is a link from the Bing search engine team blog, where they announce the release of their attempt at social networking. At it’s core, the announcement details a new search interface, which allows you to search for stuff and simultaneously ask your friends on various social networks about the stuff you find while searching. After reading the post and watching the intro videos, my initial reaction reaction is that Microsoft finally got it right.

What’s so impressive about this is what Microsoft didn’t do—they didn’t try to make their own social network and have users talk to each other on that network, and that network alone. Google tried this with Google+, which may or may not be a ghost town; I don’t know, I don’t use Google+ (rimshot). Microsoft took a look at Bing, and said, “You know what we have here? We have a pretty darn good search engine. Let’s make it a better search engine, not by having users talk to each other on our proprietary and not-yet-extant new social network, but by allowing people to talk using whatever social network they’re using now.” They’re choosing their battles. The end result is (what appears to be) a genuinely improved search engine that users can benefit from immediately.

I realize this whole post is singing praises of a tool which isn’t even completely live yet; when I load Bing, it brings me to their old search page. That being said, I’m tickled pink that they’re getting smart about choosing their battles, and I’m hopeful that the engine will, in fact, represent the next stage of internet search.

What I’ve learned about resumes, from reading way too many of them

As I mentioned in a recent post, I recently participated in a panel discussion of academics who made the move to industry. A few members of the audience asked questions related to resume writing. r the talk, I was asked to provide my list of suggestions to be distributed to the group; I present that list ips to those present. After the talk, I was asked to provide my list of suggestions to be distributed to the group; I present that list to you.

The below list is based on a combination of my own experience and that of my colleagues. Note that I’m working as a quantitative analyst in the banking industry, and these reflect what I want to see for such a position. Other jobs in other fields may (and almost certainly will) want to see other things emphasized. Still, this should be a good starting point.

(Note that this is a list of suggestions regarding your resume, the kind that is sent out out when applying for industry—and some academia—positions. This is not describing how to format your curriculum vitae, which—at least for academics—will have much more detail, including all publications, all honors, all courses taught, all students mentored, and more besides.)

Do

• Make it easily readable
• Keep it two pages or shorter
• Include the following:
  • Education (university name, degree granted, year granted, GPA if applicable)
  • Relevant work experience (Company name, location, dates, summary of what you did in short paragraph/bullet points)
    • Spend time here… this is the most useful part to a potential new boss. Describe succinctly but fully. Use full titles when describing positions.
    • Academic projects are OK, but not valued as much as relevant work
    • Irrelevant work experience is usually not necessary
  • Skills: techniques, programming languages, familiarity with specialized software. “Expert” = expert, “Proficient” = I’m very skilled at this, “Familiar with” = I’ve used this before a few times. This is NOT universal, but a general guideline.
  • Selected publications (no more than 3)
  • Honors/Awards (I’d suggest no more than 5)

Don’t

• Use words such as “Excellent”, “Superstar”, “World-class”, or other superlatives when describing yourself. It may sound good when spoken, but often comes across as plain when read. Try it yourself on the following excerpts from resumes I’ve received:
  • “PhD in hardcore mathematical research topic”
  • “Enormous leadership skills and teamwork”
  • “Top-ranked expert of … in USA”
• Say you have done projects you haven’t actually done
  • In the programmer industry, it is almost standard practice to apply for jobs requiring experience in an unfamiliar programming language, because the knowledge of the languages you do know is mostly transferrable to the new one. While this may come across as “lying”, I view this as perfectly fine, so long as you study the language before you get to the interview. Basically, so long as whatever you say is true when you come to the interview, you’re OK. Do note, however, that if you list it, it’s fair game for questions, so if you take this approach make sure you really learn it well in that short period of time.
• List coursework. We generally know what you’ve done, as we’ve done it ourselves. I guess this doesn’t apply for particularly unusual courses and highly relevant coursework, but that will be far and few between.

Meh

• You can list interests/community and volunteer work, but it’s really not required. It can give some flavor about your personality, but you almost certainly have more useful ways to utilize the space on the resume.
• Many people seem to list a “summary” section at the top of the resume. Our approach is, this won’t hurt you, but your resume is a summary, so to summarize the summary is kinda silly. Many of my colleagues just skip them altogether and just read the actual text, so you’re just wasting space.

You are NOT your college major

Colleges have a problem. They want to teach students skills that will be useful later in life. The problem is, almost every non-trivial job requires a pretty wide base of different skills. Consider the computer programmer… he has to be knowledgable about computers and computer programming, but also skilled in project management (for organizing his projects), time management skills (for reading countless manuals), problem solving (for the endless bug-fixing), and writing and communication skills (for writing documentation and talking to users). For our programmer to graduate to becoming a Project Manager, he has to not only be pretty darn good at those things, but also have good people skills and managerial talent.

In an effort to make it simpler for students to match courses to professions, colleges have created a whole variety of college “majors”, which are basically nothing more than a group of courses that, as a whole, are supposed to teach the student whatever they need to know to become familiar enough with that subject to gain employment¹. Avoiding for a second whether they have been successful in this goal, the concept of a major does have desired effect of making it more simple for students—students interested in history major in history, easy enough. However, it has resulted in students defining their knowledge base via their college major, which is a tremendous disadvantage to the student. Just to make this clear:

You are NOT your college major.

Your college major is a single phrase; “biomedical engineering”, “English literature”, “political science”. You, however, are a far richer individual than that single phrase can capture, and you should almost certainly avoid using it to characterize yourself and your skillset.

I recently participated in a panel for academics who have made the move to industry. At some point during the discussion, somone asked something similar to the following:

I’ve been doing research in for the past few years, and I’m also considering leaving academia, but I’m worried that I won’t be able to find work in . Where do I find jobs for people like me?

There seems to be a mentality amongst college graduates that since they pursued a particular college major, they are restricted to working in that particular field. This simply isn’t true. Your skillset is defined by what you know; your college major represents a minor subset of that.

Nowhere is this more important than when applying for jobs. This comes up in two ways; candidates don’t apply for jobs because of perceived ineligibility, and candidates don’t even search in the correct fields.

The first situation often presents as the following. You—the recent college graduate—will be searching through job postings, and you’ll see something which looks appealing and which you think you may be a qualified candidate, but it’s asking for someone with a degree in <not your major> with years of experience. You think, “gosh, I don’t have that!”, and you don’t apply. The second situation is even more tough; you’ve done a lot of work in college pursuing major X, but along the way you’ve become very familiar with Y. However, when the time comes to look for jobs, you think, “gee, I just spent however many years earning my degree in X; I guess I’d better look for jobs in that field.” It may not even cross your mind that, despite your expertise and knowledge about Y, you should look for jobs in that field; no one granted you a degree in that topic, so clearly you’re unqualified, right?

Don’t be that guy! Consider the following piece of advice I received from someone when starting my own job search just a few months ago:

A job posting is essentially a wish list. It would be nice if the perfect candidate came along, but those posting the opening know from the get-go that it’s pretty unlikely.

If you’re interested in a job, definitely take the time to apply. If you’re interested in jobs in a few different fields, that’s great! You’re multi-talented! Apply to all of them! If this means you’ll have to customize your resume for different fields, do it! I have four different resumes I use for different purposes, as do many of my colleagues. Even more fundamental, when you’re beginning your job search, spend time thinking about which fields you want to cover when searching. Not sure where someone with talents like yours should be applying? Talk to your department and try to find a list of their alumni, along with their current employer… most of them would be more than willing to meet for coffee. Ask them how they chose their current employer, where else they looked, and where their friends went. You’ll likely be surprised at how many graduates are working in completely different fields from their college major.

Above all, don’t sell yourself short. You’re a skilled, intelligent person, who has probably learned lots of things while working through college. Don’t limit your skills by thinking “I’m only a major”.

FizzBuzz for Applied Statistics

Hiring is tough. One problem particularly related to hiring for skilled positions is that many academic institutions focuses more on theory than on application, making your highly-educated job candidate almost useless in an actual non-academic business setting. Over the years, there have been some very clever techniques for determining whether a job candidate is not only smart, but also able to actually, you know, do stuff. In the world of software engineering, the FizzBuzz programming test has proven to be so effective as a screening technique for programming job applicants that it’s frankly embarrassing. Briefly, the test asks a job applicant to sketch out a blueprint of an incredibly easy programming task, one for which there are hundreds of possible correct answers. As it turns out, for any programming job opening, there are going to be dozens, if not hundreds, of completely unqualified applicants. The test has become a great way to do a first-line check as to whether the applicant is even eligible for the position, after which you can examine whether their qualifications match.

Fast-forward to today: we’ve recently begun interviewing for a new quantitative analyst position in my group. As it turns out, most of the resumes look very similar; masters or PhD in statistics/financial mathematics/something similar, little work experience. Thinking back to the Fizzbuzz concept, I wanted to create a quick test to ensure that the fellow in front of me is familiar with the basics of applied statistics. My thinking led me to this scatterplot, from a dataset I was working with at the time:

At the interview I asked each candidate the following:

Look at this plot. Don’t worry about the axes or scale; for all intents and purposes, it’s just X and Y.

1. What are some of the interesting features of this plot?
2. How would you predict Y given X?

(I frequently format my speech into a numbered list.)

The good thing about these questions is that there is no correct answer; as with the FizzBuzz question, there are hundreds of ways to approach the problem. However, you have to have some basic applied knowledge regarding dealing with datasets. Sure, your thesis work on applying Bayesian methodologies to sparse highly dependent datasets may have been very ingenious, but if you can’t work with data, well, come back when you can.

I’ve had mixed responses to this question. Everyone notes that the data is clustered. Some comment that there’s a slightly bimodal distribution (x=0 and x=1). One or two have discussed the apparent independence of the axes. Regarding the prediction task, most provide the more “obvious” answers, such as simple data transforms and linear/nonlinear regression techniques. Some refer back to techniques they’ve used in past projects, research or otherwise. I have had a few people who simply didn’t know what to do and simply shrugged.

Overall, the test gives a good idea how the candidate right now would think about doing such a simple, and (in our workplace) very commonplace task. Sure, it doesn’t test for familiarity with machine learning techniques or an understanding of heteroskedasticity, but it’s not intended to. Just as FizzBuzz doesn’t test knowledge of namespaces or function templates, this is intended to be a similar benchmark, testing only the most basic applied statistical skills. The fact that a couple applicants have been tripped up makes me think it may be a useful test for others interviewing for applied statistics/data mining positions. If anyone has any similar techniques they use for determining basic applied statistics skills, feel free to share them below.

Reflections from my Future Self

The interaction between personal identity and the aging process is a fascinating one. Everyone has had some experience in which remembering some personal experience brings about a sense of detachment, as though the person in the memory “isn’t really them”, because their current self is smarter, wiser, and more experienced than that awkward fellow in their memory. I’ve found this to be particularly true when remembering past decisions I’ve made; many times I’ve looked back at the thought process that led to my choice with a mixture of amusement and embarrassment, a sort of fond incredulousness at my stupidity back then. “If I knew back then what I know now,” I tell myself, “I would have been waaay better off, and that decision would have gone much better.”

Or would it? There’s no real lower limit as to how long ago something had to have happened before you can reminisce about it. There have been times where, just a few minutes after I made a decision, I was already deep into the old “oh what an idiot was I” routine. Oftentimes, wisdom acquired through experience comes with a heavy cost, and reminiscence of stupid decisions is a painful, yet oddly addictive, tonic.

A few years ago, while agonizing over some decision (which seemed so important at the time, but now I can’t even remember what the topic was), I realized that I could extend the exercise beyond just my previous selves. Who’s to say, for example, that the decision I’m about to make right now isn’t one I’ll be thinking about in five minutes with that familiar mixture of derision and bemusement? Right now, I am an idiot compared to my future self, who will have learned a good deal through the mistake I may be about to make. Extending this even further, the decisions I’ll make tomorrow are subject to the same thought pattern. No matter what I do, I know that I’ll be subjected to an “audit”, so to speak, by my future self, and there’s no one I want to disappoint less than myself.

This perspective has given me two different outlooks on each decision I make. Firstly, it makes me realize that despite how much I’ve looked into whatever I’m about to do, experience will be the ultimate judge. Some people don’t need that reminder, but it helps keep me humble. Secondly, this perspective is a driving force for me to continually tackle new projects. The worst possible scenario I can imagine is where my past self looks at my future self and says, “You’re a lot stupider than I expected you to be. What have you been doing?” I want my future self to have the knowledge due to him from hundreds of attempted projects and countless decisions. I owe it to myself - literally - to make my future self as smart as I possibly can.

Thinking about thinking

Quote of the day:

“Thinking is a skill, and can be improved.”

(From the cover of Six Thinking Hats by Edward De Bono.)

What an amazing concept. Who thinks about thinking? My first thought was, I’m too busy thinking to think about thinking. Considering the idea, though, I doubt my “thinking process” is that good that it can’t use improvement. Considering some more, my thinking process probably stinks. Some suggestions for other ways to work on the skill of thinking:

• Play sudoku.
• Play Blokus.
• Play Set.
• Do a crossword puzzle.
• Participate in Project Euler.
• Build legos.
• Draw.
• Write a blog post.

Choose one of these things, and do it as infrequently as once a week. Each of these requires you to use a different type of skill set, and only by doing the activity will you improve your overall ability to think. I know I’ll be trying to work on improving how I think.

Google’s “Evil” Pivot

Take a moment to consider Google in early 2000, at the beginning of their ascendancy to becoming a household term, before they had gotten into the online advertising business. At that point, if they had made it standard practice for them to munge up enhance their search results with some of the other products they offered, no one would have batted an eyelash about the “evilness” of it. It’s a company making a business decision; they’re using one tool to promote another, and that’s a completely standard business move. Product integration, as everyone knows, is usually a “good thing”.

The obvious difference here is that integration was very clearly not a “good thing” for Google, which is why they didn’t do it. Search for Google was something beyond pristine; it was sacrosanct. “Don’t be evil”, for years, meant that Google Search would first and foremost be the most effective search tool available on the web. The prevailing attitude at that time was, “If it could bring in profits, all the better, but for goodness sake, don’t mess with the quality of our search results.” Even the lure of higher profits took a backseat to tampering with search results; that’s why they didn’t mix search with all their other products, even though there was plenty of opportunity to do so. Imagine if, when searching for some appliance, they only showed those vendors who supported Google Checkout. That integration would have meant more business, but a worse search experience. At the start, Google wasn’t willing to do that.

Then Adsense came along. Sure, Adsense didn’t “affect” search results — they’re right there in front of you! — but suddenly your search results aren’t just “search results”, they’re “search results, with something very likely relevant, but not necessarily most relevant, as the very first link”. Was this a bad business move? Of course not; ads now account for the vast majority of their revenue. However, this was the beginning of a shift away from pure search.

And then Facebook happened.

Maybe it was the ads; the money advertisers will spend on search ads, where relatively little is known about the user, is peanuts compared to what they’ll spend when you know the user’s name, age, gender, location, friends, family, education, proportion of time spent online, pasttimes, etc. Maybe it was the user interaction; people search for a second and then click away from Google, whereas they’ll sit on Facebook for embarrassingly long periods of time. Maybe it was simply that they felt that social networking was too big to ignore. Whatever it was, Google switched from being a search company which also offers email and spreadsheets and analytics and other stuff, to being a social networking company which also does search and other stuff.

Is this a good idea? That depends on your definition of “good idea”, and how you parametrize it. However, there’s no denying that when a huge pivot like this happens, the original product will almost certainly suffer. I guess Google’s decided they’re fine that, ten years from now, the colloquial term for searching may be “just Bing it”.

Always take the front-row seat

To paraphrase the immortal words of Cookie Monster, “networking is an always thing.” There’s no secret formula that, when applied, creates an awesome network for you. Like any other process, it takes a lot of small steps, and there are many tools you should use.

One opportunity which I find that far too few people use applies to lecture settings. Far too often, when some visitor—often a subject matter expert—comes to give a presentation about whatever topic, people tend to sit together with friends in the back of the room. If you’re attending this talk, it’s because the guy speaking has something to share with you, and you probably would stand to benefit from talking to him. Sitting up front means the guy/girl speaking will see your face more, and may increase the likelihood that you’ll get to talk face-to-face after the lecture, particularly if the person is going to get quickly mobbed. Most of all, though, if you sit up front you’ll be noticed as That Guy Who Sits Up Front. That can mark you as an interested person, which can be good all around.

Is this a very minor suggestion? Yes. However, this requires zero effort on your part. Give it a shot; it may help you end up meeting someone. I’d love to hear your other simple networking tips in the comments!

“Reading Doth Not A Programmer Make” (or, How To Effectively Learn To Program)

I recently had to quickly familiarize myself with the R statistical computing language for a new job. Being a veteran of the “learn new languages and immediately forget them” school of thinking, I wanted to make sure that this one stuck.

For R, the problem definitely isn’t a shortage of learning material; there are literally hundreds of online tutorials on how to use it. There are entire books that are devoted to individual components of the language. The problem is that, to steal a phrase from some dead Latin-speaking person, “reading doth not a programmer make.” To really become an expert programmer one simply has to program. The best way to do this is to have a project that you’re going to complete in the new language; by going through the planning, coding, and debugging phases, you’ll experience the nuances of the new environment; novel design patterns, different coding conventions, and all those subtle ways that a new language can excite and infuriate you. After you do that first project, you really don’t know that much—after all, you’ve only finished a single project—but you’re familiar enough that you could probably adapt to a new project quickly, and your subsequent questions will be less “what’s the def statement mean?” and more “why did you use design pattern A instead of B in this function?”.

The downside to the “do a medium-sized project” approach is that it may take more time than you have; in my case, I needed to learn a new language in about a week, while working on other projects. At most, I was able to dedicate two hours a day to learning the new language. I spent a good deal of time going through code from previous projects others on my team had written, but I found that it had similar retention to reading; simply reading through code, while helpful for seeing design patterns, is not enough for learning a new language.

For this project, a different approach was needed, and one that has been in use by teachers since time immemorial. A google search for the term “Introduction to R course” unearthed this website, which contained numerous homework assignments meant for the students. The assignments turned out to be written pretty well; each assignment contained what was, in effect, two small analysis projects. The homeworks, in effect, proved a good proxy for my “medium-sized projects” which I needed to learn the R; after completing them, I was by no means an expert, but I was able to converse intelligently about the language, and I was ready to begin really getting into the meat of the language. I had, in effect, progressed beyond the “beginner” phase, which was what I had been hoping to accomplish.¹

There are two specific lessons I learned from this. From the standpoint of a student, the best way to learn a new language is to use it. How you use it during the learning phase is not as important—it’s far more important that you simply get real-world experience writing scripts and programs, so you have the necessary exposure to the language. From the standpoint of the teacher, though, the take-home is different; the best homeworks are the ones that get the student to use the language. Even better, have the student use the language for different purposes, to expose them to the many different facets of the language. A well-written homework assignment greatly enhances the overall learning experience.

Application Request: Simplified SSH key generator

It seems to be a well-accepted fact that public key authentication provides significantly more security than password-based authentication [1][2][3][4]. Unfortunately, for most people, every time we want to set up public key access for a particular server (which occurs pretty rarely), we do the following:

1. Google “set up SSH key access”
2. Follow the steps, including:
   1. Typing yes to some scary question about a Man-in-the-middle attack
   2. Sending keys back and forth, hopefully using something more secure than FTP

How hard could it be to put a good UI on all this to make it simple? Most of it is very simple; all the work would be in the UI.

In that vein, this post is a call to the general developer community, requesting a simple app for setting up keys. The app I’m envisioning would broadly follow this “spec”:

1. It should only require the remote server URL and login credentials as input
2. Running the program should:
   1. run the ssh-keygen command
   2. put files where they belong
   3. ideally, figure out what the appropriate host key is for the server¹
   4. copy the appropriate files to the remote server
   5. tell me when I’m done

I’d pay for it.