Data management for the stressed out academic: this is why we can’t have nice things.

Things have been crazy for the last couple weeks. I’ve been working with a visiting scientist on a really awesome model, as well as chipping away at my Big, Big Data Proposal… wrangling collaborators for the latter, tracking down data and piecing it together for the former.  Everything, of course, happens all at once, right? As you can imagine, my computer became a flurry of open documents, spreadsheets, databases, scripts; and my desk covered in post-its with single thoughts, random equations, constants, papers with a single line highlighted. Basically, I’ve become my own worst data (and workflow) management nightmare.

After breathing into a paper bag for a bit, I collapsed Friday night with the resolution to devote the first half of next week to cleaning up the mess I’d made. Well, it’s Tuesday now, and there are more random bits everywhere.*  Dangit.

So where am I going with this? I suppose it’s  to let you know, gentle reader, that, in data management, as in life, things gets off the rails all the time.  This doesn’t mean you’re hopeless (at least I hope not). I’m fairly confident that I can get everything I’ve done back on track. But I’m going to say one thing- if  my data management heading into this had been what it was even a couple years ago, this hole I dug myself would be at least twice as deep to get out of.  Oh, the things I wish I hadn’t been too intimidated to try sooner!

The most important saving graces have been:

1) having my master data in a database, which allows me to see fairly easily what my different queries were, and how that affected the outcome of my analysis

2)scripted analyses.

3)Glitter pens**

So, stick with me, friends. I’m hoping to emerge out the other side of this next push a better data manager, maybe even a better data manager with a really cool funded research program and a nice high profile pub or two. ***


*Bad data manager! BAD! 
**not for color coding anything. They just make my notes-to-self and to-do list less intimidating. The reason I was afraid to use them for years was that I feared the mockery of my peers. But since I got my PhD, I’ve decided to just own it. Hi, I’m Christie and I like glitter pens.
***let’s not forget fame and fortune.


About cbahlai

Hi! I'm Christie and I'm an applied quantitative ecologist and new professor. I am an #otherpeoplesdata wrangler, stats enthusiast, and, of course, a bug counter. I cohabitate with five other vertebrates: one spouse, one first grader, one preschooler and two cats.
This entry was posted in Uncategorized and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s