Thursday, September 18, 2014

Hate to Love? Or Ignorance to Interest?

I have to say I am not a very steady person, or rather I tend to hold bad or weird views and then have to change them. Like playing monopoly with my brother. To start a game without the expectation of having a chance to win is madness, that or you're throwing the game right off the bat. But soon, despite my belief that I could possibly win, he gets more and more properties and I have less and less as I try to cut losses. So if I were the steadfast type I would play the next 30 minutes to an hour and let him win completely and utterly, and be crushed in my monopoly belief. So I quit and go do something more useful with my time.

I am definitely more steady, or steadfast, in some of my views, Christianity, environment, and so on. Those have changed on occasion, but take much more time and thinking, much more than realizing I am losing at monopoly. But with this post I specifically want to sort out my feelings about data.

You, the few, ask: Data? What is he on about?

I used to say: Anything but data, give me anything but tables and input! I was a Computer Engineer, not a computer scientist mucking about in gobs and gobs of data! Computer Engineers don't have anything to do with data, that's the furthest thing from electronics and my brand of software. Right, not wrong in a moral sense, but definitely shortchanging myself by poo-pooing what I thought was too far away from electronics.

Did I hate data? No, but I wore my ignorance on my sleeve. By the way I still need to wash my sleeves of all the ignorance I display unless to rightly say, 'I don't know.' We certainly dealt with small amounts of data in my labs in college, lab reports required data. I also took a Digital Signal Processing course, but of course that was real-time, might I be forgiven that I never saw the loads of data, just constructed filters?

So becoming a Software Engineer at the Joint Astronomy Centre I thought that I was data free. I could work on objects that only made data for other people to look at, except error reports. That became bothersome quite quickly, error, fix, error, fix, design, test, error, error, error. Keep iterating that, essentially just trying to keep things running. When one of my colleagues asked if I could help him plot out some engineering data I jumped at the chance.

I wasn't thinking 'Oh no, data!' Rather I was just happy to get away from the hum-drum (or really a bad process) and try to help track down a problem with a system rather than just responding to its faults. And let me tell you about the data we collected: It showed a trend. Trends are a double edged sword, but they cut better than just busting one error at a time through blunt force trauma. The thing with trends is that they can show you a direction that the data is going, and you can respond to it. Opposing that is that trends are just like fashions, in fact those are used together quite often. Something trending one way one day may switch directions without much notice.

However, even with this potential fickleness a physical system such as a telescope doesn't often fix itself, so tracking a trend and figuring out what variables are effecting it is doable. Even if it does seem to be fixed, looking for the variables in data around the time it fixed itself could give us the information to fix the system before it unfixes itself.

And boy do I love manipulating data, though I am bad about the creative ways of visualizing it. So I am glad when my misconceptions are challenged. And when I think my "truths" have been challenged I should think about those too.

You can drown in data, but if you learn to swim it can change everything you see.

Featured Post


John studied himself in the mirror as best he could through tears. Red, puffy eyes stared back at him, a running nose already leaked just a ...