Week 6: Course challenge | Process data from dirty to clean
1. Scenario 1, questions 1-5
You are a data analyst at a small analytics company. Your company is hosting a project kick-off meeting with a new client, Meer-Kitty Interior Design. The agenda includes reviewing their goals for the year, answering any questions, and discussing their available data.
Before the meeting you review the About Us tab on their website and their business plan, linked below:
Meer-Kitty Interior Design has two goals. They want to expand their online audience, which means getting their company and brand known by as many people as possible. They also want to launch a line of high-quality indoor paint to be sold in-store and online. You decide to consider the data about indoor paint first.
To use the template for the survey feedback, click the link below and select “Use Template.”
Link to template: Kitty Survey Feedback
OR
If you don’t have a Google account, download the file directly from the attachment below.
When you refer to the Meer-Kitty survey feedback tab, you are pleased to find that the available data is aligned to the business objective. However, you do some research about confidence level for this type of survey and learn that you need at least 120 unique responses for the survey results to be useful. Therefore, the dataset has two limitations: First, there are only 40 responses; second, a Meer-Kitty superfan, User 588, completed the survey 11 times.
As the survey has too few responses and numerous duplicates that are skewing results, what are your options? Select all that apply.
- Repeat the survey in order to create a new, improved dataset.
- Talk with stakeholders and ask for more time.
- Remove the duplicates from the data and proceed with analysis.
- Locate another dataset about indoor paint.
2. Scenario 1 continued
During the meeting, you also learn that Meer-Kitty videos are hosted on their website. For each product offered, there is an accompanying video for customers to learn more. So, more views for a video suggests greater consumer interest.
Your goal is to identify which videos are most popular, so Meer-Kitty knows what topics to explore in the future. Unfortunately, Meer-Kitty has just three months of data available because they only recently launched the videos on their site.
Without enough data to identify long-term trends about the video subjects that people prefer, what should you do?
- Tell the client you’re sorry, but there is no way to meet their objective.
- Find an alternate data source that will still enable you to meet your objective.
- Watch the videos and use your gut instinct to identify which are most successful.
- Move ahead with the data you have to determine the top video subjects.
3. Scenario 1 continued
Now that you’ve identified some limitations with Meer-Kitty’s data, you want to communicate your concerns to stakeholders. In addition to insufficient video trend data, your main concern with the indoor paint survey is that the data isn’t representative of the population as a whole.
Clearly, one particular respondent, the superfan, is overrepresented. This is an example of margin of error.
- True
- False
4. Scenario 1 continued
The stakeholders understand your concerns and agree to repeat the indoor paint survey. In a few weeks, you have a much better dataset with more than 150 responses and no duplicates.
To use the template for the survey feedback, click the link below and select “Use Template.”
Link to template: Kitty Survey Feedback
Or, if you don’t have a Google account, download the file directly from the attachment below.
If you are using the template, please refer to the New Meer-Kitty survey feedback tab. You notice that questions 4 and 5 are dependent on the respondent’s answer to question 3. So, you need to determine how many people answered Yes to question 3, then compare that to responses to questions 4 and 5. That way, you will know if questions 4 and 5 have any nulls.
You decide to use a spreadsheet tool that changes how cells appear when they contain the word Yes. Which tool do you use?
- Filtering
- Data validation
- CONCATENATE
- Conditional formatting
5. Scenario 1, continued
You have finished cleaning the data to ensure it is complete, correct, and relevant to the problem you’re trying to solve. Then, you complete the verification and reporting processes to share the details of your data-cleaning effort with your team.
Your team notes one aspect of data cleaning that would help improve the dataset. They point out that the new survey also has a new question in Column G: “What are your favorite indoor paint colors?” This was a free-response question, so respondents typed in their answers. Some people included multiple different colors of paint. In order to determine which colors are most popular, it will be necessary to put each color in its own cell.
You use a spreadsheet function to divide the text strings in Column G around the commas and put each fragment into a new, separate cell. In this example, what are the commas called?
- Substrings
- MIDs
- Delimiters
- Partitions
6. Scenario 2, questions 6-10
You’ve completed this program and are interviewing for a junior data scientist position. The job is at B.Spoke Market Research, a company that analyzes market conditions using customer surveys and other research methods. The detailed job description can be found below:
So far, you’ve had a phone interview with a recruiter and you’ve secured a second interview with the B.Spoke team. The recruiter’s email can be found below:
You arrive 15 minutes early for your interview. Soon, you are escorted into a conference room, where you meet Jodie Choi, the data science lead. After welcoming you, the behavioral interview begins.
For your first question, your interviewer wants to learn about your experience with spreadsheets. She says: Sometimes the team needs data that is stored in different spreadsheets. So, we use spreadsheet functions to help us find the information we need.
What function would you use to search for a certain value in a spreadsheet column to return the corresponding piece of information?
- RETURN
- SEARCH
- COUNTIF
- VLOOKUP
7. Scenario 2, continued
Next, your interviewer wants to know more about your understanding of tools that work in both spreadsheets and SQL queries. She explains that the data her team receives from customer surveys sometimes has many duplicate entries.
She says: Spreadsheets have a great tool for that called remove duplicates. But when writing a SQL query, what command should you include in your SELECT statement to remove duplicates?
- DIVERSE
- DIFFERENT
- DISCRETE
- DISTINCT
8. Scenario 2, continued
Now, your interviewer explains that the data team usually works with very large amounts of customer survey data. After receiving the data, they import it into a SQL table. But sometimes, the new dataset imports incorrectly and they need to change the format.
She asks: What function would you use to convert data in a SQL table from one datatype to another?
- CAST
- CHANGE
- CONVERSE
- COALESCE
9. Scenario 2, continued
Next, your interviewer explains that one of their clients is an online retailer that has a vast inventory. She has a list of items by name, color, and size. Then, she has another list of the price of each item by size, as a larger item sometimes costs more. The stakeholder needs one list of all items by name, color, size, and price.
She then says: In situations such as this one, could you use the CONCAT function to add strings together to create new text strings?
- Yes
- No
10. Scenario 2, continued
For your final question, your interviewer explains that her team often comes across data with extra leading or trailing spaces.
She asks: Which function would enable you to eliminate those extra spaces? You respond: To eliminate extra spaces for consistency, use the TRIM function.
- True
- False
11. Scenario 1, questions 1-5
You are a data analyst at a small analytics company. Your company is hosting a project kick-off meeting with a new client, Meer-Kitty Interior Design. The agenda includes reviewing their goals for the year, answering any questions, and discussing their available data.
Before the meeting you review the About Us tab on their website and their business plan, linked below:
Meer-Kitty Interior Design has two goals. They want to expand their online presence, which means getting their company and brand known by as many people as possible. They also want to launch a line of high-quality indoor paint to be sold in-store and online. You decide to consider the data about indoor paint first.
To use the template for the survey feedback, click the link below and select “Use Template.”
Link to template: Kitty Survey Feedback
OR
If you don’t have a Google account, download the file directly from the attachment below.
When you refer to the Meer-Kitty survey feedback tab, you are pleased to find that the available data is aligned to the business objective. However, you do some research about confidence level for this type of survey and learn that you need at least 120 unique responses for the survey results to be useful. Therefore, the dataset has two limitations: First, there are only 40 responses; second, a Meer-Kitty superfan, User 588, completed the survey 11 times.
As the survey has too few responses and numerous duplicates that are skewing results, you should remove the duplicates and continue analyzing the remaining 29 responses.
- True
- False
12. Scenario 1 continued
During the meeting, you also learn that Meer-Kitty videos are hosted on their website. For each product offered, there is an accompanying video for customers to learn more. So, more views for a video suggests greater consumer interest.
Your goal is to identify which videos are most popular, so Meer-Kitty knows what topics to explore in the future. Unfortunately, Meer-Kitty has just three months of data available because they only recently launched the videos on their site.
Without enough data to identify long-term trends about the video subjects that people prefer, what are your available options? Select all that apply.
- Move ahead with the data you have to determine the top video subjects.
- Watch the videos and use your gut instinct to identify which are most successful.
- Ask to wait for more data and provide Meer-Kitty with an updated timeline.
- Talk with Meer-Kitty stakeholders and ask to adjust the objective.
13. Scenario 1 continued
The stakeholders understand your concerns and agree to repeat the indoor paint survey. In a few weeks, you have a much better dataset with more than 150 responses and no duplicates.
To use the template for the survey feedback, click the link below and select “Use Template.”
Link to template: Kitty Survey Feedback
OR
If you don’t have a Google account, download the file directly from the attachment below.
If you are using the template, please refer to the New Meer-Kitty survey feedback tab. You notice that questions 4 and 5 are dependent on the respondent’s answer to question 3. So, you need to determine how many people answered Yes to question 3, then compare that to responses to questions 4 and 5. That way, you will know if questions 4 and 5 have any nulls.
You decide to use a spreadsheet tool that changes how cells appear when they contain the word Yes. When using this tool, what is the word Yes?
- The value in a VLOOKUP statement
- The value in a conditional formatting rule
- The value in a CONCATENATE range
- The value in the COUNTA range
14. Scenario 2, questions 6-10
You’ve completed this program and are interviewing for a junior data scientist position. The job is at B.Spoke Market Research, a company that analyzes market conditions using customer surveys and other research methods. The detailed job description can be found below:
So far, you’ve had a phone interview with a recruiter and you’ve secured a second interview with the B.Spoke team. The recruiter’s email can be found below:
You arrive 15 minutes early for your interview. Soon, you are escorted into a conference room, where you meet Jodie Choi, the data science lead. After welcoming you, the behavioral interview begins.
For your first question, your interviewer wants to learn about your experience with spreadsheets. She says: Sometimes the team needs data that is stored in different spreadsheets. So, we use a spreadsheet function to find the information we need.
There is a spreadsheet function that searches for a value in the first column of a given range and returns the value of a specified cell in the row in which it is found. It is called SEARCH.
- True
- False
15. Scenario 2, continued
Next, your interviewer wants to know more about your understanding of tools that work in both spreadsheets and SQL. She explains that the data her team receives from customer surveys sometimes has many duplicate entries.
She says: Spreadsheets have a great tool for that called remove duplicates. Does this mean the team has to remove the duplicate data in a spreadsheet before transferring data to our database?
- Yes
- No
16. Scenario 2, continued
Now, your interviewer explains that the data team usually works with very large amounts of customer survey data. After receiving the data, they import it into a SQL table. But sometimes, the new dataset imports incorrectly and they need to change the format.
She asks: Is there a SQL function that can convert data types such as currency, dates, and times in a SQL table?
- Yes, data types including currency, dates, and times can be converted.
- No, only currency can be converted.
17. Scenario 2, continued
Next, your interviewer explains that one of their clients is an online retailer that needs to create product numbers for a vast inventory. Her team does this by combining the text strings for product number, manufacturing date, and color.
She asks: If you encountered a situation where you wanted to add strings together to create new text strings, which SQL function would you use?
- COMBINE
- COALESCE
- CREATE
- CONCAT
18. Scenario 2, continued
For your final question, your interviewer explains that her team often comes across data with extra leading or trailing spaces.
She asks: Which SQL function enables you to eliminate those extra spaces for consistency?
- TRIM
- LEN
- SUBSTR
- LENGTH
19. Scenario 1 continued
Now that you’ve identified some limitations with Meer-Kitty’s data, you want to communicate your concerns to stakeholders. In addition to insufficient video trend data, your main concern with the indoor paint survey is that the data isn’t representative of the population as a whole.
Clearly, one particular respondent, the superfan, is overrepresented. This means the data doesn’t represent the population as a whole.
When surveying people for Meer-Kitty in the future, what are some best practices you can use to address some of the issues associated with sampling bias? Select all that apply.
- Increase sample size
- Use data that keeps updating
- Use data from only one source
- Use random sampling
20. Scenario 1, continued
You have finished cleaning the data to ensure it is complete, correct, and relevant to the problem you’re trying to solve. Then, you complete the verification and reporting processes to share the details of your data-cleaning effort with your team.
Your team notes one aspect of data cleaning that would help improve the dataset. They point out that the new survey also has a new question in Column G: “What are your favorite indoor paint colors?” This was a free-response question, so respondents typed in their answers. Some people included multiple different colors of paint. In order to determine which colors are most popular, it will be necessary to put each color in its own cell.
You decide to use a spreadsheet function to divide the text strings in Column G around the commas and put each fragment into a new, separate cell. You are using the SPLIT function.
- True
- False
21. Scenario 2, continued
Next, your interviewer wants to know more about your understanding of tools that work in both spreadsheets and SQL. She explains that the data her team receives from customer surveys sometimes has many duplicate entries.
She says: Spreadsheets have a great tool for that called remove duplicates. In SQL, you can include DISTINCT to do the same thing. In which part of the SQL statement do you include DISTINCT?
- The UPDATE statement
- The SELECT statement
- The FROM statement
- The WHERE statement
22. Scenario 2, continued
Now, your interviewer explains that the data team usually works with very large amounts of customer survey data. After receiving the data, they import it into a SQL table. But sometimes, the new dataset imports incorrectly and they need to change the format.
She asks: Is there a command or function that converts data in a SQL table from one datatype to another? You respond: Yes, it’s the CAST function.
- True
- False
23. Scenario 2, continued
Next, your interviewer explains that one of their clients is an online retailer that has a vast inventory. She has a list of items by name, color, and size. Then, she has another list of the price of each item by size, as a larger item sometimes costs more. The client needs one list of all items by name, color, size, and price.
She then asks: If you were to use the CONCAT function to complete this task, what would it enable you to do?
- Search for and return missing products in inventory
- Create a unique key to tell products apart
- Clean the product identifier text strings
- Create a new product database table
24. Scenario 2, continued
For your final question, your interviewer explains that her team often uses the TRIM function when writing SQL queries.
She asks: What is the TRIM function used for in SQL?
- To eliminate extra leading or trailing spaces
- To return the smallest numeric value from a list
- To shorten the list of results
- To eliminate null values
25. Scenario 1, questions 1-5
You are a data analyst at a small analytics company. Your company is hosting a project kick-off meeting with a new client, Meer-Kitty Interior Design. The agenda includes reviewing their goals for the year, answering any questions, and discussing their available data.
Before the meeting you review the About Us tab on their website and their business plan, linked below:
Meer-Kitty Interior Design has two goals. They want to expand their online audience, which means getting their company and brand known by as many people as possible. They also want to launch a line of high-quality indoor paint to be sold in-store and online. You decide to consider the data about indoor paint first.
To use the template for the survey feedback, click the link below and select “Use Template.”
Link to template: Kitty Survey Feedback
OR
If you don’t have a Google account, download the file directly from the attachment below.
When you refer to the Meer-Kitty survey feedback tab, you are pleased to find that the available data is aligned to the business objective. However, you do some research about confidence level for this type of survey and learn that you need at least 120 unique responses for the survey results to be useful. Therefore, the dataset has two limitations: First, there are only 40 responses; second, a Meer-Kitty superfan, User 588, completed the survey 11 times.
As the survey has too few responses and numerous duplicates that are skewing results, you decide to repeat the survey in order to create a new, improved dataset. What is your first step?
- Delete all of the data from the current, skewed survey.
- Write new, improved survey questions.
- Find a survey tool that only allows someone to complete the survey once.
- Talk with stakeholders, explain the new timeline, and ask for approval.
26. Scenario 1 continued
Now that you’ve identified some limitations with Meer-Kitty’s data, you want to communicate your concerns to stakeholders. In addition to insufficient video trend data, your main concern with the indoor paint survey is that the data isn’t representative of the population as a whole.
Clearly, one particular respondent, the superfan, is overrepresented. What does this situation describe?
- Sampling bias
- Statistical significance
- Margin of error
- Confidence level
27. Scenario 1 continued
The stakeholders understand your concerns and agree to repeat the indoor paint survey. In a few weeks, you have a much better dataset with more than 150 responses and no duplicates.
To use the template for the survey feedback, click the link below and select “Use Template.”
Link to template: Kitty Survey Feedback
OR
If you don’t have a Google account, download the file directly from the attachment below.
If you are using the template, please refer to the New Meer-Kitty survey feedback tab. You notice that questions 4 and 5 are dependent on the respondent’s answer to question 3. So, you need to determine how many people answered Yes to question 3, then compare that to responses to questions 4 and 5. That way, you will know if questions 4 and 5 have any nulls.
You decide to use a spreadsheet tool that changes how cells appear when they meet a certain value — in this case, the word Yes. You are using VLOOKUP.
- True
- False
28. Scenario 2, questions 6-10
You’ve completed this program and are interviewing for a junior data scientist position. The job is at B.Spoke Market Research, a company that analyzes market conditions using customer surveys and other research methods. The detailed job description can be found below:
So far, you’ve had a phone interview with a recruiter and you’ve secured a second interview with the B.Spoke team. The recruiter’s email can be found below:
You arrive 15 minutes early for your interview. Soon, you are escorted into a conference room, where you meet Jodie Choi, the data science lead. After welcoming you, the behavioral interview begins.
For your first question, your interviewer wants to learn about your experience with spreadsheets. She says: Sometimes the team needs data that is stored in different spreadsheets. So, we use a spreadsheet function to find the information we need.
There is a spreadsheet function that allows a data analyst to search for a value in the first column of a given range and return the value of a specified cell in the row in which it is found. What function allows you to complete these tasks?
- VLOOKUP
- SEARCH
- COUNTIF
- RETURN
29. Scenario 1, continued
You continue cleaning the data. You use tools such as remove duplicates and COUNTIF to ensure the dataset is complete, correct, and relevant to the problem you’re trying to solve. Then, you complete the verification and reporting processes to share the details of your data-cleaning effort with your team.
While reviewing, your team notes one aspect of data cleaning that would improve the dataset even more. They point out that the new survey also has a new question in Column G: “What are your favorite indoor paint colors?” This was a free-response question, so respondents typed in their answers. Some people included multiple different colors of paint. In order to determine which colors are most popular, it will be necessary to put each color in its own cell.
What spreadsheet function enables you to put each of the colors in Column G into a new, separate cell?
- Divide
- Delimit
- SPLIT
- MID