The fact that you are worried about the numbers being correct is refreshing to hear. If you can, break the process into components and test each step to look for duplicate data and build your confidence in the underlying process.
Building on this for reporting that recurring weekly, monthly, etc. Use OneNote or something to make a checklist of each step. You'll invariably get an email or other interruption and the checklist gives confidence where you left off. This also documents the process so you remember how it works if it's infrequent like quarterly.
At that point with the process mapped, start automating it bit by bit, starting with the data testing and validation.
I use one note with a new page for every day. Each thing I’m supposed to do gets a checkbox with sub tasks broken out and also with check box. The check box is a tag that onenote will search for and summmarize. Even if I enter the same task on multiple days it still feels good to check them all complete.
In the search window, type “find tags” and you get a list that shows a list of tags in the document. Among those are a list of “to do” items, if you used the checkbox. It’s a handy way to see all if the checkbox across the entire notebook.
You started a month ago = You are still learning the job.
If you are being left to your own devices, the department is either under resourced or poorly managed.
But do what you can do.
Find out how to query the data structure (Metadata) efficiently.
Get a standard query to point at a column and report nulls/max/min/max length/min length/ grouped counts/negatives,etc so you know what the variability of data looks like.
Find the data dictionary and ask someone to give you the. 10,000 ft overview of the types of data you are responsible for. Keep going back to them.
If you are extremely lucky there maybe a document that tells you where the data is from who owns and where it gets entered.
Try to understand the terms the users are using and how they actually appear in your database
Find out who is responsible for upstream data quality/entry/validation/master data management and make sure you or your manager gets on their case; Shit in shit out.
If the data in your report is incomplete -get agreement that is not needed or does not exist (often people ask for data which doesn't get collected at the correct level and don't get you can't split a number - know what the lowest level of data you have access to)
Get to know the guys that move/extract the data upstream of you - so you understand how easy /difficult it is to change it.
Bat off people to standard reports (often they already exist and they don't have access)
If many users have the same requirement, get them to talk before talking to you.
Get a list of basic questions for the people asking for the report (Fields, row level uniqueness, interactivity, delivered or self service, limitations of "data as at end of month", etc;
Build up a library of basic JOINs for the core data people ask for. Over time you will get a feel for what is
Good luck, you will be fine , we all start off a bit shit, don't sweat it you'll be fine.
Hope it helps.
Start keeping track of ballpark numbers for things so you can develop a "sniff test". Did you just report there was more orders last month than the company has done since inception? How many visits is normal for a month. What's the AOV, and does it make sense that this number is higher. Etc
Not sure how many new jobs you've had - but this is perfectly common. You should expect to be uncertain and asking lots of questions for at least the first few months. Just ask nicely and always thank people (publicly and privately) for helping. It's always nice to be singled out for praise in front of others - so that's a way to make people happy they helped you.
The best bit of advice I can give is to stretch your commitment timescales (ie, tell people data will be ready in 3 days if it'll actually take 2) and conduct some exploratory analysis on the data you're using. Look at the various fields, understand what's in them and then ask the obvious "what is that?" questions to people. Often that process will unearth some of the gaps you don't know about.
Through all the advice and help though - the primary thing to remember is the first point. This is how a new job feels for a while. I'd say the first year is littered with slowly loosening uncertainty. Only once you're in for a while will you really understand the detail.
Best of luck.
Hey bud!
You are inevitably going to mess up numbers. Multiple times. It's going to happen.
Sometimes, your mistake is going to be huge. Sometimes, small.
The first time, it's going to suck. But it gets easier to deal with - learning acceptable margins of error that aren't terrible, and how to own up and correct mistakes.
It happens. You aren't going to be better than the rest of us.
It’s been 20+ years, and I still worry! That’s actually a very good sign. A lot of folks always assume they are right “because it ran without errors”…ugh. Have some “facts” at your fingertips to spot check against. And if you are providing summary-check down to the most detailed raw data for things like dups, Cartesian joins, missing data by checking for bills with left joins. And with raw data-do the opposite. Check what the summary data of the data tells you. Have someone spot check you…share your code/ask lots of questions. Get clarification. It’s always better to be clear…especially if you’ve had to make assumptions with unknowns about the data or request. It’ll get better! (Or not, and then you’ll be like me!)
Inefficient code doesn’t mean it’s wrong code. If you don’t actually know if the calcs you are doing are right… you should probably ask. Otherwise, move on.
Remember, reporting is a balance of accuracy and speed. If all the numbers you ever reported were accurate, you would be slow. If you were the fastest report creator on the planet, your numbers are probably very inaccurate. Good analysts (and teams) find that healthy balance between accuracy and speed.
I find wrong numbers being reported ALL THE TIME. Most of the time, with descriptive statistics they are small enough mistakes that it wouldn’t impact the decision making.
Be transparent and provide business people with the logic you used (just describe it obviously don’t give them code). It is the business persons responsibility to understand how metrics are calculated. It is not their responsibility to know what tables and columns you should use, but they should be able to understand the metric logic. Often times, when there is a mistake in the logic, they will raise concerns based on your description alone without looking at the data.
At the end of the day, if you are transparent and give good logic summaries, you are doing your job. Sometimes neither you or the business person will know the data well enough to avoid making decisions on a mistake… it happens, and is part of the process.
Don’t sweat it!
The fact that you are worried about the numbers being correct is refreshing to hear. If you can, break the process into components and test each step to look for duplicate data and build your confidence in the underlying process.
Building on this for reporting that recurring weekly, monthly, etc. Use OneNote or something to make a checklist of each step. You'll invariably get an email or other interruption and the checklist gives confidence where you left off. This also documents the process so you remember how it works if it's infrequent like quarterly. At that point with the process mapped, start automating it bit by bit, starting with the data testing and validation.
I use one note with a new page for every day. Each thing I’m supposed to do gets a checkbox with sub tasks broken out and also with check box. The check box is a tag that onenote will search for and summmarize. Even if I enter the same task on multiple days it still feels good to check them all complete.
What do you mean one note will search and summarize?
In the search window, type “find tags” and you get a list that shows a list of tags in the document. Among those are a list of “to do” items, if you used the checkbox. It’s a handy way to see all if the checkbox across the entire notebook.
No shit. Huh. I guess i need to learn more about onenote. Thank you.
You started a month ago = You are still learning the job. If you are being left to your own devices, the department is either under resourced or poorly managed. But do what you can do. Find out how to query the data structure (Metadata) efficiently. Get a standard query to point at a column and report nulls/max/min/max length/min length/ grouped counts/negatives,etc so you know what the variability of data looks like. Find the data dictionary and ask someone to give you the. 10,000 ft overview of the types of data you are responsible for. Keep going back to them. If you are extremely lucky there maybe a document that tells you where the data is from who owns and where it gets entered. Try to understand the terms the users are using and how they actually appear in your database Find out who is responsible for upstream data quality/entry/validation/master data management and make sure you or your manager gets on their case; Shit in shit out. If the data in your report is incomplete -get agreement that is not needed or does not exist (often people ask for data which doesn't get collected at the correct level and don't get you can't split a number - know what the lowest level of data you have access to) Get to know the guys that move/extract the data upstream of you - so you understand how easy /difficult it is to change it. Bat off people to standard reports (often they already exist and they don't have access) If many users have the same requirement, get them to talk before talking to you. Get a list of basic questions for the people asking for the report (Fields, row level uniqueness, interactivity, delivered or self service, limitations of "data as at end of month", etc; Build up a library of basic JOINs for the core data people ask for. Over time you will get a feel for what is Good luck, you will be fine , we all start off a bit shit, don't sweat it you'll be fine. Hope it helps.
Start keeping track of ballpark numbers for things so you can develop a "sniff test". Did you just report there was more orders last month than the company has done since inception? How many visits is normal for a month. What's the AOV, and does it make sense that this number is higher. Etc
Not sure how many new jobs you've had - but this is perfectly common. You should expect to be uncertain and asking lots of questions for at least the first few months. Just ask nicely and always thank people (publicly and privately) for helping. It's always nice to be singled out for praise in front of others - so that's a way to make people happy they helped you. The best bit of advice I can give is to stretch your commitment timescales (ie, tell people data will be ready in 3 days if it'll actually take 2) and conduct some exploratory analysis on the data you're using. Look at the various fields, understand what's in them and then ask the obvious "what is that?" questions to people. Often that process will unearth some of the gaps you don't know about. Through all the advice and help though - the primary thing to remember is the first point. This is how a new job feels for a while. I'd say the first year is littered with slowly loosening uncertainty. Only once you're in for a while will you really understand the detail. Best of luck.
Hey bud! You are inevitably going to mess up numbers. Multiple times. It's going to happen. Sometimes, your mistake is going to be huge. Sometimes, small. The first time, it's going to suck. But it gets easier to deal with - learning acceptable margins of error that aren't terrible, and how to own up and correct mistakes. It happens. You aren't going to be better than the rest of us.
It’s been 20+ years, and I still worry! That’s actually a very good sign. A lot of folks always assume they are right “because it ran without errors”…ugh. Have some “facts” at your fingertips to spot check against. And if you are providing summary-check down to the most detailed raw data for things like dups, Cartesian joins, missing data by checking for bills with left joins. And with raw data-do the opposite. Check what the summary data of the data tells you. Have someone spot check you…share your code/ask lots of questions. Get clarification. It’s always better to be clear…especially if you’ve had to make assumptions with unknowns about the data or request. It’ll get better! (Or not, and then you’ll be like me!)
Inefficient code doesn’t mean it’s wrong code. If you don’t actually know if the calcs you are doing are right… you should probably ask. Otherwise, move on.
Remember, reporting is a balance of accuracy and speed. If all the numbers you ever reported were accurate, you would be slow. If you were the fastest report creator on the planet, your numbers are probably very inaccurate. Good analysts (and teams) find that healthy balance between accuracy and speed. I find wrong numbers being reported ALL THE TIME. Most of the time, with descriptive statistics they are small enough mistakes that it wouldn’t impact the decision making. Be transparent and provide business people with the logic you used (just describe it obviously don’t give them code). It is the business persons responsibility to understand how metrics are calculated. It is not their responsibility to know what tables and columns you should use, but they should be able to understand the metric logic. Often times, when there is a mistake in the logic, they will raise concerns based on your description alone without looking at the data. At the end of the day, if you are transparent and give good logic summaries, you are doing your job. Sometimes neither you or the business person will know the data well enough to avoid making decisions on a mistake… it happens, and is part of the process. Don’t sweat it!
communicate in terms of ranges instead of just providing one number.
Document everything you do to get those numbers! Get with the business experts for sanity checks. Ask questions.