Wednesday, December 05, 2018

Fair Warning

If you ever want to enjoy a movie, radio show, or anything with bird sounds in the background, don’t ever learn bird calls and bird songs. I’ve said this before, either in discussion or maybe even somewhere on this blog. Actually, in counterpoint, when they get it right it really helps set the scene. If you want to start learning bird songs then try this Bird Academy class. Or if you are already somewhat experienced you could try the Quiz.

This quiz that I speak of is actually the project I have been working on the last three months. It is a quiz consisting of mostly unrated media, photo or audio, from the Macaulay Library archives, most of it contributed through ebird. I know it’s sort of like my child, but I absolutely adore it. I already took enough quizzes that I was mostly familiar with 80% of the birds in Japan in May…

I really would like to jump into the spaced repetition, but if as one of the rules in the link says you have to make your own cards, then it would be really hard for us to do with our random species and asset selection. It isn’t totally random, you can pick a place and a time of year. We get a list of likely birds seen in that part of the world, you will be disappointed with Barrow, Alaska in December, and then pick 20 random species. Then we get the photo or audio clip with the least number of ratings and serve up the quiz.

This works well for the most part for well inoculated users, there are terrible tiny photos and misidentified photos, but in general I really can’t get enough of it myself. There are a few issues that I think would make this a better experience for someone trying to use this to learn a new region: Weight the chance to see a species by the number of times you have seen it previously, and try to never show the same photo to the same user if it can be helped.

Let’s say that you are trying to learn birds of the Northeast US. One of them is most certainly the American Robin. At the moment in my area there are getting to be around 200 potential species. If you take the quiz 10 times and see 200 photos then you might have seen the good ol’ Robin about five or six times, and other species on an average of two or three.

This is a several runs of completely random choosing of species for quizzes. At 10 quizzes the user will only have seen 125 species of the 200. Even at 50 quizzes there are still 4 species the user has seen once, though because of long tails and randomness 40 quizzes should be a better indication at 13 species seen less than twice, or rather no chance to learn the species.

So totally random is not really a great way to go. On the other hand totally not random is an algorithmic nightmare unless handed over to Machine Learning or better: Machine Teaching. But let’s say we need a bit of middle ground. How about just calculating a weighted random selection that favors the species a user hasn’t seen yet? Basically every time the user sees a species, they are a bit less likely to see it. It isn’t a perfect solution, but it would be that much better.

This second approach shows a much steeper decline in the number of unseen species. The tail is still potentially long, but we have a much better chance of seeing more species over the course of fewer quizzes. We are only missing repetition for 57 species after 10 quizzes rather than 75 for the completely random approach. I know this doesn’t sound like much, but 20 species could be significant, especially if they are the most common species.

The second issue is a bit more tame. Essentially we are sorting the possible assets by how many ratings they have and then taking the most recent one with the least ratings. All we really need to do is get rid of the most recent and check for a user’s previous rating on a piece of media. It would take more time, but it could make it that much more rich. Really this is only a problem for species that have very few photos, so go take more! Maybe also unpopular bird species don’t get enough attention in that there isn’t enough turnover to churn-up the number of ratings on a regular basis. Hopefully will see less of the latter with more participants.

Why am I geeking out about this? Well it goes back to when my wife was studying birds for her two jobs doing bird surveys. She hand-built study decks and then reviewed them every night with Anki. Of course that fits in with the spaced repetition from before, but then I started working at the lab and saw how many photos we had, what a treasure trove to build something that could build a set of flashcards from 26k+ Robins, etc.! Starting this was like a dream come true, and through some not-so-dreamy work here we are. It needs some tweaking, but it is still superb.


I wrote this back in March before we released the app. The whole idea of the app is to help Macaulay Library to curate data via community ratings. Of course it is pretty close to being a learning app that people can learn birds with, but ML doesn’t want to step on toes. Nine months in we have collected 3 million ratings which helps us to show some of the best photos the community has taken. Ratings were critical in the release of eBird species pages where 6k previously un-curated species had photos and audio chosen if they had any.

I think the biggest wish now is to add a bit of filtering. For instance people might be well-versed in their finches, but want to focus on their shorebirds, even if you are birding locally. Currently with hundreds of species it could take quite a few quizzes to learn all of your shorebirds, though we would get quite a few ratings in the process. It is certainly a possibility, but we have plenty of projects to keep us busy, even for a good part of next year.

I've taken a few more since the start.

Tuesday, December 04, 2018

Conflicted Progress

Read these two articles:

How can these two exist at the same time? One must cancel out the other like antimatter/matter annihilation. Or can they both be right?

Interestingly, I think both are possible, though the “Superpowers” article is mainly writing from the viewpoint of survivorship bias. That is the author is saying that because they did it, so can you, and because you are reading it then they succeeded wildly by following their own rules.

The Farnam Street Blog looks at several cases of Winner Takes All, but doesn’t tell you how to overcome that, except maybe buying several thousand copies of your own book. Does it cancel out the rule-following concept in the more positive article? No, but I think it tempers it quite a bit.

When it comes down to it: We can all improve, maybe even the top performers, but really the only person that would have thought we have superpowers is our former self. Let me frame it with the Recency Effect.

Let’s say that you are a hotdog eating contestant and you can eat ten hotdogs in a minute. You are so good that you have sponsorships and tour the country. Then someone comes along and introduces a new technique and is able to eat one hundred hotdogs in a minute. At the behest of your sponsor you learn the technique and are able to eat one hundred plus one hotdogs, reclaiming your crown.

Being at the top and then reclaiming the top certainly looks like superpowers to folks lower in the pecking order, but even the lightweight packing away only eighty per contest would scoff at your old record of ten due to your new achievement. You might appreciate the difference, but you only really have your initial reaction to gauge just how monumental a jump that is.

All’s good in hotdog-land, but what about starting from the bottom like removing a bad habit? Initially people may notice if it is an obvious habit, but soon it will be old history. Good changes are like a differential for people noticing and hopefully a integral in improving life. That is: people will only really notice as the change is happening, or if asymptotic like a haircut, right after, but then that will be the new normal. But maintaining a good habit like brushing your teeth might literally improve your life expectancy, and no one runs away from conversations anymore.

Self-set rules will only get you so far in becoming the best, the rest is how they relate to your goals, but I think they are a huge step in getting you away from the worst. Rule-following has another name: discipline.

Monday, December 03, 2018

And It's December

I just turned in my final assignment. By final I mean final assignment, meaning the last one in my master’s degree. I mean like if my team doesn’t miss more than four out of 16 points I will graduate on the 15th. I mean final like there is a chance for extra credit that is bigger than any other assignment that might mean the difference between me graduating and not. I dislike this class with a passion.

The degree, Master’s of Computer Science at Georgia Tech. The final class, Introduction to Health Informatics. Would I ever suggest taking this course? No, just a straight no. The only reason I can see for someone to take it would be if their job is going to deal with Health Informatics. If they are already dealing with it, forget it, they will be happier not taking this class.

For those of us that were never going to go into healthcare software this is really just an exercise in frustration. The paramount frustration is supplied by the course itself, or rather the original instructor. He essentially describes all the different ways everyone is doing healthcare data, and what a nightmare interoperability is. The tone is set in frustration that even after 60+ years of thinking about electronic health records that it’s still a really hard problem. Just like telling doctors they aren’t doing something right.

The second frustration was how the class was run. It seems that it was mostly TA run because the professors aren’t actually technical, but having taken classes by David Joyner I found their expectations did not normally match their directions and the grade weights for particular sections that were administrative details far exceeded actual content. Missing a name in a presentation was the difference between getting 100% and getting a B in one case.

The third frustration was with our TA, responding to pertinent questions after the assignment was due. Nothing better to make an assignment even tougher.

If there was an alternative, I would have taken it. And throughout the 4 years that I have labored through all of these courses there have been a few that have been promised to whole time, one that would have been interesting is the computational journalism or something like that. It couldn’t have been worse than this one, right?

In some ways I really shouldn’t complain, the main one being that there were no tests. A few courses with tests have been my worst courses. Give me coding and projects over tests, I am just not built for them anymore, who is with google right there?

The amount of time and money that I will have available in the future is sort of daunting. What will I do with all that time? But I think saving the money is going to be pretty obvious for that. I guess I can rewrite my novel, write another, and spend time with my wife. She’ll like that.

Wednesday, August 22, 2018

Last Class!

One last class! I am a bit apprehensive, but since a large chunk is a group project and I already have seven group members I think that we will be able to do okay. I am not entirely sure what role I should take in the group, everyone responded to my post so it feels like I am coordinating, but I don't necessarily want to be the Project Manager.

I think I see everyone as having two roles, one of PM, Quality Assurance, Video Editor (for our presentations), and then everyone is a developer with a range of different modules/tasks. I think we can start talking about this and maybe formalizing our approach for the next month before we get or choose a project.

It is unlikely that I will post much during this period, hopefully next time I will have a Master's of Science in Computer Science.

Wednesday, March 14, 2018


I will not complain about my proposal being failed just to get my attention, I will not. I certainly need to fix it even though it is better than most of the ones that I read and got some useless feedback about how it was better than the reviewers’.

Alright, but I do need to complain, you know, righteous indignation and all of that. This week I received a credit card promotion from my own bank. Probably because I still have a cash-back card and they are tired of giving me barely a penny on a dollar. They want to transition me to a card that is “better.” Great, but that is only a small grievance, the real sticker is the copy text at the top of the promotion: “Use your Bank-name-here credit card to help pay down debt.”

Um, no. A credit card is not used to pay down debt. A credit card is used to accumulate debt.

Let’s get biblical for a second, debt or assurance of debt is a chain around your neck that can be yanked by the person that is owed. Go check out Financial Peace University. Although we don’t ascribe to the entire system described by FPU it is certainly closer to what we practice than not. Just average credit card debt is a bit disturbing. Maybe we should look at getting rid of it. With only a house to pay for at the moment it is sort of silly to have a monthly accumulation as well. Unfortunately for the simplicity of not racking up debt to be paid off monthly debit cards are not as well protected as credit cards, hmmm…

I started and finished Ready Player One by Ernest Cline so here’s a review. Overall: Not a terrible story, but talk about IP nightmare for the upcoming movie. I am a geek, nerd, whatever, but the level of 80s pop-culture was way beyond my ability to cope, even recognizing 60-80% of it. But here is where it is ultimate:

The disconnectedness from the real world was certainly surreal, but then was conveniently forgotten at the end. There are so many points in the story where it could be an interesting delve into futurism or even keeping self-consistent and then it just throws in an 80s reference or several pages of exposition and then brings us back to the moment reeling. Some moments stand out in my mind, but maybe because they are so few. On top of that all the victories are essentially hollow, there is existential crisis throughout, but it doesn’t seem to affect any performance on challenges.

Would I suggest you read it? Was it a waste of time? Meh on both counts. If you are a super fan of the 80s then this book was written for you, but it wasn’t even a waste for me because I can see what not to do in my writing.

How about that Black Panther movie? It was pretty good, I wouldn’t jump straight to the best in the Marvel universe, but only because it is really hard to compare all of them side-by-side. That said, it did talk about some issues that translucent-skinned individuals like me don’t often think about. Well done, well told, worth seeing.

Saturday, February 24, 2018

How to Compound a Rainy Day

Watch the solar output when it’s super dark and rainy; it’s like watching your investments tank on a bad market day, but with even less ability to control it. Yeah, but it certainly compounds a sunny day, or even better: mitigates having to stay inside during an incredible day. Okay, barely.

This week I dove into the world of not Numpy with Java, ND4J. I didn’t even get into the ‘N’ of ND4J just keeping to a one dimensional array. Let’s just say that it has many really nice options for dealing with arrays. I will not mention the fact that you cannot easily get an array back to an otherwise usable list in Java without some machinations.

I think that is what I miss most about python in general. Even if a library didn’t have something very specific for a certain community it’s likely the community had someone willing to dive in and make a module that can do it and then posted it. Java certainly has its libraries and all the people using it, but somehow it’s a different community.

When someone asks how to do something there are ten different answers and they must all be encapsulated with a class. Sure if I want to do this again I will move it to a function. If it becomes widespread and is more like an object then I might think about making it a class, but seriously. And then spring… I just want to curl up with some nice classy class based views in Django and leave the mess of Spring behind.

In other news, I ran twice this week. Tuesday and Wednesday were up in the 60s and this morning wasn’t too chilly at 40. If New York only has 2 months of real winter, and it was cold, I guess it’s better for my running. Cloudy, but keeping at around 60% of electricity used produced by solar, ran, and now off to clean something and write my class paper proposal. Content.

Tuesday, February 13, 2018

The Only Odd One is Me

Not to stir up any fears, but here we go. That in-the-moment feeling of everyone watching, while everyone else says they are not. Can we be real for a second? If you are not staring at a screen in a crowded place, how observant are you? Do you notice when someone does something, at least innocuous such as tripping on flat ground and then looks around to see if anyone saw? Yeah, sorry, I saw that one. Shall we just share a smile and keep going forward?

The problem is that there are so many non-innocuous situations that anything from dumb to malevolent. Going to this World Life Expectancy site can be a bit eye opening, plenty of malevolence, but plenty of stuff that I could classify as dumb, but really it is just sad. Not sad as in “those people should have seen it coming,” rather maybe it could have been prevented by better/earlier treatment or just better friends. Maybe in my age group the traffic accidents could be helped by smarter cars

Anyway, there is a point to this: Whatever I am suffering that is not acute AND terminal, there are possibly a few thousand people suffering of the same thing. That is true of post-graduate online masters degrees. Of artfully procrastinating by practicing writing. Of making bad lists with weird sentences. Though that last might be just me at the moment. But there are people suffering, or...  Oh, how odd am I?

I am odd, there are so many things that I am bringing to the front of my mind through biases where I haven’t really met many people that do, or have done, a permutation of different things that I have. This is the deal with permutations, even if someone else has done everything I have, but flipped the order of two events, then by definition we are not the same, because we experienced those two events with different views. Twins can be a prime example.

You are odd as well. I have found out, somewhat too recently, that I have no idea what is going on inside your skull and trying to understand every nuance is impossible and/or maddening. I spend so much time with my wife, yet it is better to have a model than to be “sure” of what she is thinking.

What frustrates me more is how odd I feel in trying to make people understand my thoughts. I think my thoughts are pretty good, the problem lies where your thoughts conflict with mine. Either I am too dismissive because I am too absorbed by my thoughts, or I have already considered it thoroughly and have a ready answer. Though that seems dismissive too because I cannot explain it well enough.

Four years of theater should have made me a better actor, if not in hiding what I think, at least in having the poise to articulate it better. Well no, I get cut-off when I need a moment to speak, but then ramble when someone should just cut me off. Can I have it both ways? Thinking, listening, and responding. How might I even practice that? How can I even find the place to practice that?

I would love to tell stories too, but I would for now settle with schooling on conversation. I have to say that I write better than I did back in this post, the first, from twelve years ago. Even so, I can improve my writing. Better to practice than to pine about not having practiced. Only a recent addition, that, maybe it will stick.

No one is odd in that everyone is odd. Maybe I will find some more odd people that we can be mostly mutually accepting of one another.

Tuesday, January 23, 2018

Wait, Prepare, Wait

What is probably the worst combination in the history of the world? Promising and procrastinating.

I do this all the time, I promise my wife I will do something, but if I don’t do it right away I forget, or just plain procrastinate. I would like to say that procrastination is doing other things while still having the thing that you need to do in the forefront of your mind, but then I think forgetting about things has just the same effect, although less stressful until you remember. Of course the disappointment in myself is pretty big, but that is dwarfed by the feeling of having disappointed someone else, especially my wife.

Right now, as I write this, I am procrastinating. Sort of. I’ve read through assignment 3 and I need to let it simmer. But I am also waiting for the peer feedback for assignment 2 to be uploaded so that I can get that done for the class. It doesn’t look good for “the drop” tonight. I am also using this moment to write, write “I” enough that I think I might need to edit this.

This is an okay equilibrium at the moment, because it has many layers of preparedness: I am early wanting peer feedback, I have read the assignment a week before it is due and it builds off of the previous one, I am practicing writing which is crucial for this class, and I will update my to-do list before going to practice my cello, then reading and then bed. Right now, everything is under control.

It’s a good feeling, but of course fleeting because this week is going to flit by one way or another.

So is this waiting any better than just getting things done at the last instant? It’s hard to say from this end. Is it temporary motivation that I am chomping at the bit? Have I finally begun to learn discipline? (Note: whenever I think this way I remember all the things that I would like to add to my disciplined schedule and how many things I have failed at already in terms of discipline.)

It’s going to be hard to say from the other end. There is something to finishing something all in one go. Something satisfying about having just finished, and it’s addictive (Wait But Why.) I remember one of my most exhilarating coding days, I knew I needed to write a user interface to test the new receiver in the telescope. I put it off until the day of, and finished, not well, that same day. Here is the description in passing. It made me realize that being an expert at a chosen technology can mean that you can get things out quickly. While that is an awfully nice thought, a better one is getting it done over the course of weeks, consulting with people, and being on-hand for the commissioning.

I still have a problem with discipline at work as distractions are plentiful, but I also can get quite a bit done just being that much more of an expert. Something about having worked for 10 years as of the 21st of January.

There is no perfection in my work. I strive to understand better ways of doing things, new things I come across, but I am much more confident. I wait quite a bit more, but on the other side. I find things to do, things that I should be remembering sooner, and things that may make at least one person’s job easier.

Sunday, January 14, 2018

Star Wars: The Last Jumble

What do you get with “space bombers” actually dropping bombs on another ship? A terrible premise that falls apart before the movie even really begins. And then a low-speed chase? Anybody with a grain of sense would have just jumped out a few destroyers and then jumped back in just on top of the rebel fleet, knocked out engines or power, and let the rest of the fleet catch up. Rebellion over.

Though I am too “good” hearted to have been happy with that outcome, but really those are just the starting mistake and the rest of movie’s premise really just make it hard to suspend my inner-critic, or my disbelief at the very least; it just becomes hard to enjoy any of the rest of the movie.

I have almost no problem with saving Leia from the vacuum of space, except: Both the rebels and pursuers are burning fuel, and in space that means they are accelerating, meaning that Leia would have to react instantly to draw herself toward the ship, or be left very far behind. And she would have been shredded by debris if she wasn’t vaporized in the explosion, both requiring a presence of mind where she probably wouldn’t have floated out at all. The Expanse does it better.

Speaking of starship bridges, there are so many things wrong. Why is the bridge so close to the surface of the vessel? Does the retro style of Star Wars really explain large indefensible windows? And what is anybody doing on a starship that is so easily perforated without a pressure suit and helmet to hand? I guess it would mess with the flowing gowns the costume department insists on.

I don’t like the sharing visions, it just seems like a poorly conceived or explained way to cheat.

So if you kill a character, how fast do you replace him with another rogue? About half a movie, except this guy hasn’t bought in to the rebels, so maybe that is Episode 9? At least he is more interesting than most of the rest of the characters.

And the final jumble: Force balancing. The overall implication of force balancing makes this a very fraught subject. Does it mean, for instance, that Rey should kill all other “good” force users so that she is more powerful? Don’t train anybody? Maybe they will all end up stunted and she can be the most powerful. It’s almost Sithian.

Does having two equally balanced but supremely powerful force users mean that no others exist? Oh no that womp rat is a light-side force user, but only so far as a womp rat can be, but it means Rey is that much less powerful and unequal to Kylo Ren.

Basically they just introduced a philosophy for the world of Star Wars that is nearly impossible to continue without bending over backwards in order to explain it.

To summarize I give it full marks for getting deeper into a quagmire introduced in the Prequels,  full marks for a shiny but despicable cantina, almost full marks for imaginative action sequences, and far less than stellar marks for a plot line and any adherence to even in-universe physics.

Start 2018, Review Some 2017

What’s more disappointing than failing to do well in a class? Working hard and then failing anyway, at the last irreparable moment.

I didn’t actually fail, but C’s are no longer average and certainly not in a program with thousands of computer science students. I would like to say that I am a more well-rounded individual because I did other things that led to me not putting enough time into this class. That is unfortunately not the case. I quit orchestra, said no in several cases where the church needed help with sound, stayed home from small groups, sequestered myself from visitors, and relied on my wife’s good graces to survive and stay in an up-kept house. Really there were only two main (good) things that I got from this experience:

I came up with an effective way, for me, to study for an open-note exam. The fact that I didn’t fail it outright means that I must have done something right. Essentially it breaks down into how to go back over the lectures and the slides looking and listening for key pieces, noting them, and then having some recollection of it. This review took place for the week leading up to the exam. I didn’t do great, but much better than the test that prompted me to drop this class last spring.

And, amazingly, I learned about a few things that I would have liked to have known going into the robot competition last year. Go figure. So unless they drastically change the Computer Vision Course I would suggest giving yourself between 10 to 30 hours a week in order to get things done.

Not being one to dwell on past failures, except where it is due, I am looking forward to the next class quite a bit: Education Technology. This class is much more free-form, reading, discussing, and a project/paper. There are no lectures, but it looks like it will need a bunch more reading.

It will be challenging, but I think that if I use the resources at the lab, papers and interview people, then I should be on a good path to have a good path/project for this class. In fact I am starting a project that is all about tech and education, a bird quiz!

As the year is winding down I have been thinking of how I can improve a bunch of my processes. It is pretty hard to think clearly about life and homework while in the midst of it, especially in the Dark Playground. But now with more time, some hindsight, and a few good ideas there should be some subtle ways to improve.

Fortunately my positive goals are somewhat easy to accomplish: exercise, practice cello, write. Are these New Year’s resolutions? Yes and no. At the heart they are resolutions, however, they aren’t quite so nebulous as just those three things strung together in a sentence.

Before the new year began we had nine guests out for Christmas: my folks, my wife’s folks, my brother and his wife, my wife’s brother and sister, and her great uncle. Only one person had to sleep on a cot, and he wasn’t too bothered by that. Of course one major thing had to fail for it to be complete: The pellet stove gave out and therefore wasn’t helping the boiler to heat the house. The boiler has no way to prioritize domestic water and so we had no hot water for showers. We are remedying that this year, but I am glad it wasn’t the fridge or toilets or something else. It was great having a house full of people. We won’t do it again next year, but it isn’t out of the realm of possibility for the future, especially when we finish the inside of the barn.

If all goes well 2018 will be the year that I graduate.

P.S. This year I learned about events, that is, remembering events is so much better than just buying a new thing. Avoiding the detailed discussion that I would love to have about that: I went to Brain Candy Live and paid the bunch extra to meet Adam Savage and Michael Stephens in person! It was totally cool, geek credentials are totally secure.

Also my wife took me to a Piano Guys concert in D.C.! It was awesome, if a bit loud. Both within weeks of each other. I am pretty happy with that, now just to write the next great American sci-fi novel, right? Okay, maybe I should focus on Homework.

Featured Post


John studied himself in the mirror as best he could through tears. Red, puffy eyes stared back at him, a running nose already leaked just a ...