Skip to main content

Posts

Showing posts with the label data

Missing Data?

In my time working with data, I have found an almost perfect one to one relationship between if the data is not in the files that were loaded, it will also not be in the data warehouse after the files are loaded.

900 Year Fix

One of my customers was wondering why some data wasn't showing up the other day... we checked everything and we weren't getting complete records from their information system. So we asked them to check on that and today they told me the record was dated 2924. So the good news is, if no one would have found that, it would have corrected itself in 900 years.

Even Though the Data Isn't There

Even though “they” insist that the problem is in our system and not theirs , and even though they want the data to be in our system, it isn’t there if they don’t have it in their system first. The data warehouse cannot “create” the data out of nothingness and only “regurgitates” what has been fed to it. In spite of that, it is always good (for “them”) to “throw it over the fence” instead of checking on their own system first. And if I ask for an example of what isn’t showing, it is always good to avoid supplying the sample for 4 or 5 emails and then act like “they” didn’t know I needed an example before wading through miles and miles of raw data to find their stuff isn’t anywhere in it at all.

Data Integration

"The new reverse-osmosis, trans-pivotal, extract process was created by our DI team for customers who want to "will" their data to us. It works almost 47 ½ % of the time and should decrease revenue by 23% over the next 12 minutes…"

How to Make Bad Data & Waste My Time

Keep sending the wrong file and by the way, keep opening and saving it in Excel so the numbers get corrupted. Then I can start working on it and realize I can’t get accurate data from it and so stop and ask for new files which will likely be bad too. Then make sure to deny you opened it in Excel. Then later, tell me you can’t open tab files on your computer so I can tell you to stop opening the files and just send them to me the way the vendor intended them to be. Then send me nine more emails with the last one being “when will it be ready?” Because if I get a lot of “are we there yet?” emails, it always SPEEDS up the process because answering garbage is a good use of my time. Oh yeah, then tell me something can’t be seen in the database that doesn’t actually exist—so that I can wade back through the data to tell you something you should already know about your own data. Then make sure to put the ID numbers in the wrong column in your system and then tell me they aren’t showing up in o...

Bad Data in Education

Sending assessment scores with NO identification numbers is really dumb. It might be common in the world of education (where much dumb thrives unchecked), but it is NOT a good way to keep track of data. Furthermore, pulling a student ID number out-of-your-butt, is also a poor method. These numbers mean nothing to the database. You should try using REAL numbers next time. Also bad, is sending multiple (different) students with duplicate student ID numbers. It is my understand, that each student should have a UNIQUE set of identifying numbers… And finally, when your numbers are already crap, spelling the student names in a variety of ways is also NOT helpful. Example: “William” is not the same as “Billy” to the database—and who the heck is AJ? Do you REALLY know who got which score? I mean, just guess—because it would work just as well! Excellent work educators!

Bad Programming

To anyone who doesn't know a lot about computer programming , some of it is done really stupidly. Here is an analogy to help explain what I mean. Suppose that Ford made Model T 's (which indeed they did). But now they are going to make a brand new, state of the art, modern automobile. But instead of rewriting the design, they simply write the new portion and add it to what already existed. Essentially, your new car would simply be a Model T with a shiny new outside. However it would drive down the road; and the time and effort and expense of rewriting the entire program would have been avoided. Then you could say, "Wow! see? I built this new car in an hour!" But actually you built a Model T with a new paint job. Eventually, the "duct tape" holding the new and old pieces together will start to fail. Someone can come in and try to retape it which just adds more tape to the already confusing mixture of old and new. It has now become way more ...

Bad Files

Every time somebody sends me a bad file and I have to convert it, and make it ready to load into the database, I could have already loaded it if it was a good file. Then I have to explain and ask for a new file. Then they (often) argue that they sent the right file. Then they send the wrong file three more times—even though I supplied directions about how to send it correctly. Then, eventually, I get them to stop sending the wrong file or to stop opening/saving it in Excel before sending it (and converting the numbers to scientific notation and dropping leading zeroes) and when they finally send the right file it works! Meanwhile, I could have loaded the file 7 times in the time it took. What? Oh… they sent that file to me last month. Look, it has the same file name and number of records and file size. Apparently I did this last month too. Well sorry that took so long… you see I have an awful lot of work to get done and sometimes it takes a while…

CuTRis Vows to Pound Target Data Thieves

CuTRis Vows to Pound Target Data Thieves UnAssociated Press December 19, 2013 After learning that his credit card data, along with 40 million others may have been stolen while shopping at Target, CuTRis vowed to pound the thieves for such a dastardly crime! CuTRis went on to clarify, that he would deliver the beatings personally unless the thieves were big... or if they carried guns or weapons or if they were "like really tough or something."

Data Donkey

There is a server I have to access remotely that is located in another state. I won't tell you what state, we'll just call it ND to be anonymous. The server is part of a government entity and it is so slow that I have started the rumor that it is so old that it is pulled by a donkey . I keep hoping that there might be oil money available to upgrade to a hyperactive ostrich . In the meantime, I sit at my desk--'reaching' through the network(s) trying to force data into the system as it spins 'round and 'round...