Fluent Essays

Module 3 AssignmentIn this assignment, you will conduct an analysis of an online advertising campaign using NHST and confidence intervals. The data is synthetic created by a simulation. But the data resemble what the main issues are at such an online ad campaign. The assignment is designed with two objectives in mind.You practice using a statistical software such as JMP to perform a statistical tests and compute confidence intervals.Practice critical statistical thinking that understands how variation affects our inference from the data, i.e. the more uncertainty the more uncertain the inference which is reflected in larger confidence intervals. The Excel sheet posted contains the data. Provide an analysis of the data following the guide posted here and using the report template. The template is a word document that can be edited. Follow the example report for more details. Prepare the report using the following formatting guidelines:1 page, single-spaced report using 0.5 margins and two-column formatInclude title of report, then FirstName, LastName, ISDS course #, Assignment #, date (00/00/00). No title page!10 pt Font Calibri or Times New RomanJustified as sample reportCreate headings for each sectionList any references used (e.g. Module 1 Resources)Include a title for your report and create headings for each sectionInclude supporting relevant figures from the analysis in your Appendix on 2 pages only.Submit as pdf with filename first name initial last name and assignment number (for instance HSchneider#1) Be sure to review the Assignment Rubric and Assignment Example attached below. assignment_3_guide.pdf assignment_3_template.pdf assignment_3.xlsx rubric_3.pdf Unformatted Attachment Preview Assignment 3 – Guide: AB Testing applying Hypothesis Testing Assignment 3 The marketing department of ABC company proposes two ads, ad A and ad B. The CEO wants the marketing department to evaluate the two ads in a pilot study to assess the pros and cons of two online ads. A pilot study was completed and data were collected for both ads. You are asked to complete an analysis of the results and to make recommendations. The data include six columns. The first column indicates which ad was (randomly) offered to a visitor of a news website. The second column is a sequential number for the views. There were a total of 4,472 views, 2,236 for each ad. The third column indicates if a visitor clicked at an ad. The fourth column indicates if a visitor purchased products. The fifth column indicates the total revenue from a purchase for a visitor. For instance, the first 5 viewers of Ad A did not click on the ad, the 6th viewer of Ad A clicked on the ad but did not buy anything, the 10th viewer of ad A clicked on it and purchased goods in the amount of $506.39. BADM7020 if a 5\% discount were offered in ad B. This exercise is added in this assignment so that you practice the expected value and variance laws, specifically, haw does adding a constant are multiplying by a constant change the confidence intervals. You do not have to compute them but you must explain. Part of the assignment is to practice hypothesis testing, part is to get some experience with using a statistical software such as JMP to analyzing data and conduct a hypothesis test and part is practicing the laws of variance and expected values. Before you start the assignment review Hypothesis testing again and 95\% confidence intervals. Specifically, how do you compute a 95\% confidence interval for means and what information do you need to do that. The following provides some guide for using JMP to do the statistical calculations for the hypothesis testing. Here are the steps for the analysis in JMP. Open JMP and click on the “Open Folder” icon to import the Excel data into JMP. Find the spreadsheet file and click to import. Note: You also can import the data Figure 2: JMP Home Window Figure 1: Data Table Revenue is assumed to be based on the price of the products purchased before any shipping costs, advertising costs or discounts are applied. This is the amount received from the customer for the purchases. It is important to recognize that only the number of views is fixed, while the number of clicks, the number of purchases and the amount purchased are random variables. Hence, only statistics based on number of views are straightforward easy to compute. After you addressed all the questions 1-10 management wants to know how the results are affected by a $5 per click paid to Google and if the company covered the charge of $5 per shipping and what the effect would be directly from Excel using the Data Table in the JMP menu in Excel. The JMP tab is at the right of all your other menus. Figure 3: JMP tab in Excel 1 Assignment 3 – Guide: AB Testing applying Hypothesis Testing The following describes how to use the JMP platforms to do the assignment. There are two platform the Fit Y by X and Fit Model. The fit Y by X allows you to test if a difference between the outcome for ad A and B are different from zero. The Fit Model allows you to get individual means and confidence intervals for Ad A outcome and Ad B outcome. The outcomes (Y) are Click_Ad, Purchase, and Revenue. The X variable is always Ad. Now let’s look at a few examples. The first example is using the Fit Model platform for answering question 1 in the assignment below. Questions to answer in your report Provide an analysis of the problem at hand including assessments of. For each of the questions provide the estimates, the hypothesis testing result, i.e., statistically significant different from zero, yes or no and the p value. Also report the 95\% confidence intervals. 1. The click rate. What are the click rates for the two ads, i.e., what percentage of visitors click on an ad? What are 95\% confidence intervals for each of the two the click rates. 2. Are the click rates statistically significant different? If yes, at what level of p? Compute a 95\% confidence interval for difference, which is called the ad effect. 3. What are the purchase rates for the two ads based on views and what are the 95\% confidence intervals? 4. Are they statistically significant different? If yes, at what level of p? What is a lower 95\% confidence interval for the difference. 5. What are the purchase rates for the two ads based on clicks and what are the 95\% confidence intervals? 6. Are purchase rates for the two ads based on clicks statistically significant different? If yes, at what level of p? (Hint: Use Click_ad=1 in the local filter.) 7. Compute the revenue per purchase per ad and 95\% confidence intervals. (Hint: Use Fit Model and the local filter nominal Purchase=1) 8. Compute the average difference in revenue per purchase. (Hint: Use Fit Model and the local filter nominal Purchase=1) 9. Compute the revenue per view for each ad and a 95\% confidence interval. (Hint: Use Distribution with Ad in the by field.) 10. Compute the average difference in revenue per view between ad A and B with a 95\% confidence interval. (Hint: Use Fit Y by X with revenue as Y and Ad as X.) Now suppose management wants to know how costs and discounts would affect the means and confidence intervals. Total net income (revenue – costs listed below) from each of the ads. (Hint: these are NOT to be computed in JMP but BADM7020 by hand after you entered the details in the summary table listed below. To do that you need to first compute the number of clicks for each ad and the number of purchases for each ad. These are obtained by using the total views times the rates.) Take into consideration that a. costs $5 per click, b. $5 for shipping and handling paid by the company c. 5\% discount for ad B. 11. How does the discount affect the confidence intervals? Fill in the row in Table 12 of the appendix in the sample form. Hint: Use the expected value and variance laws. 12. What do the confidence intervals look like for the net revenue per view taking into account shipping and handling and discount? Use the expected value and variance laws. 13. What is a confidence interval for the Google advertising costs? Use the expected value and variance laws. Example solution for #1&2: To answer question 1 we want to compute what the click rates are for Ad A and B. To get the individual click rates we can use Analyze>Distribution. Figure 4: Distribution Platform Not that Click_Ad is nominal and in the Y filed and Ad is in the By field. The following results are displayed for ad A after clicking the Red Triangle and selecting Confidence Interval and 95\%. Figure 5: Output from the Distribution Platform 2 Assignment 3 – Guide: AB Testing applying Hypothesis Testing See the Report form posted at the assignment page to see how to report the results. To answer question 2 we need to perform as statistical hypothesis test to determine if the click rates are different. For that we test the hypothesis that the difference between the two click rates is zero or different from zero. We use the Fit Y by X platform to do that. See Figure 6. Note that we changed Click_ad from nominal to continuous to do that. If you leave Click_Ad as nominal you will not be able to get the result. BADM7020 difference is obtained from the output as 9.47\%. When testing a difference we are mainly interested in a one sided lower 95\% confidence limit that shows us with 95\% confidence that the true mean difference in click rates is at least 4.57\%. For #3 and #4 you have to do the same thing for purchase rates. For #5 and #6 you have to use a local filter for adding the condition of Click_Ad=1 as shown in Figure 8. This is because you want purchase rate based on clicks. The local filter is available for most of the applications in JMP under the top red triangle of the output. Figure 6: Fit Y by X Platform The output seems to just have a chart. But the Fit Y by X platform allows us to perform hypothesis 2-sample ttests by clicking on the red triangle and selecting t-test. The output is shown in Figure 7. The average difference in click rates between ad A and B is 0.0702 or 7.02\%. To do a hypothesis test we look at the P value (note that JMP uses Prob>|t| for the p-value of the two-sided t-test). The p is smaller than 0.0001. That means we reject the null hypothesis that there is no difference between the click rates of ad A and B in favor of the alternative that the click rates are different. Figure 8: Local Filter For # 7 and #8 you use the same platforms as for #5 and #6 but use a local filter for a nominal Purchase=1. This is because you want revenue per purchase. For # 9 and #10 you use the same platforms as for #3 and #4 without using a local filter. Figure 7: Output for t-test Now we want to compute a 95\% confidence interval for the difference. The upper 95\% confidence limit for the For #11-13 you have to use Excel or a hand calculator. They are simple applications of the expected value and variance laws. Knowing that CI are approximately (mean+/- 2*STE), you have to figure out how adding and multiplying a random variable affects the mean and STE. Note these are computed based on views. Why? 3 Assignment 3 – Guide: AB Testing applying Hypothesis Testing BADM7020 Computations for #11-#13. Revenue/view with discount: You computed already revenue per view. So how does the 5\% discount effect the mean and CI? Check the to see how a factor affects the mean. Check the laws of for the variance and see how a discount factor effects the variance and thus how it affects the STE which is the square root of the variance. This will give you the Revenue/view with discount. Revenue/view with Sh&H and discount: You computed already the mean and CI for Revenue/view with discount. Now you have to adjust the these for S&H cost/view. To do that you use the CI for purchase per view you computed already and multiply by the shipping costs. This gives you the SH&H cost per view. Now you have to combine the two confidence intervals. These are again applications of the laws of Expectation and Variance and STE. Since the revenue and purchase decision are independent we can use the sum of the squares of the STE for each of the two CI and take the square root to get the combined STE of Revenue/view with Sh&H and discount. You can compute the STE for each CI by using STE=(UCL-mean)/2. This is so because the upper confidence limit UCL=mean+2*STE. Then you can use the combined mean and add and subtract 2 times the new STE. Cost of clicks/view: Use the CI for clicks per view and what each click costs to compute the CI for the cost of clicks/view. This is an application of the expectation and variance laws. Net revenue/view: This again requires that you combine the CI for Revenue/view with Sh&H and discount: and Cost of clicks/view. Compute the STE for each, square them and add them together assuming Clicks and revenue are independent and then take the square root. This gives you the STE for the combined confidence interval of Net revenue per view. Then compute the mean and +/- 2*STE. 4 First & Last Name Assignment 3 – AB Testing applying Hypothesis Testing Business Understanding (Use this word document as Template) A/B testing Explained. What is it used for? How does it work? XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Data Understanding: Describe the data in your own words. Pay attention to what is random (uncertain) and what is fixed. XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Analysis Provide an analysis of the problem at hand including assessments of click rates, purchase rates and revenue. Click Rate (1): The click rate for ad A is 19.28\% with a 95\% confidence interval for the true mean as (17.69\%-20.96\%) and the click rate for Ad B is 26.30\% with a 95\% confidence interval of (24.51\%-28.16\%) (See Table 1). Click Rate Difference (2) The click rates are statistically significant different at the p<0.0001. The average difference in the click rate is 7\%. A 95\% lower confidence limit for the ad effect is 4.5\% (See Table 2). Purchase Rate Based on Views (3): The purchase rate for ad A based on views is ______ and a 95\% confidence interval is (____-___). The purchase rate for ad B based on views is _____ and a 95\% confidence interval is (____-____. (see Table 3) Purchase Rate Difference Based on Views (4): The purchase rates based on views are (not) statistically significant different at p=. The average difference in the click rate is ___. A 95\% lower confidence limit for the ad effect is ____ (Table 4). Purchase Rate Based in Clicks (5): The purchase rate for ad A based on clicks is ______ and a 95\% confidence interval is (____-___). The purchase rate for ad B based on clicks is _____ and a 95\% confidence interval is (____-____. (Table 5) Purchase Rate Difference Based in Clicks (6): The purchase rates based on clicks are (not) statistically significant different at p=____. The average difference in the click rate is BADM7020 Spring 2020 ___. A 95\% lower confidence limit for the ad effect is ____ (Table 6). Average revenue based on purchases (7): The average revenue for ad A based on purchases is ______ and a 95\% confidence interval is (____-___). The average revenue for ad B based on purchases is _____ and a 95\% confidence interval is (____-____. (Table 7) Average Difference in revenue based on purchases (8): The average revenue based on purchases are (not) statistically significant different at p=____. The average difference in the average revenue is ___. A 95\% lower confidence limit for the ad effect is ____ (See Table 8). Average revenue based on views (9): The average revenue for ad A based on views is ______ and a 95\% confidence interval is (____-___). The average revenue for ad B based on views is _____ and a 95\% confidence interval is (____-____. (Table 9) Average Difference in revenue based on views (10): The average revenue based on views are (not) statistically significant different at p=____. The average difference in the average revenue is ___. A 95\% lower confidence limit for the ad effect is ____ (See Table 10). Effect of cost per click, shipping costs and discounts. Describe how the costs per click affects the revenue, how the shipping cost effects revenue and how a discount affect revenue. Provide a summary table (Table 12) and explain. In the last column of Table 12 indicate (Y/N) if there was statistically significant difference at p<0.05. Fill out each row of Table 12. Note that the last four rows are obtained by hand calculations. Conclusion The conclusion should recap what you found in your analysis. What are the pros and cons of the different metrics such as per view and per click, or per purchase? Explain the different conclusion drawn from: Purchase per view: Why is that useful? Purchase per click: What does it tell you? Revenue per view: Why is that useful? Revenue per purchase: What does it tell you? What do the different metrics tell you? Which analysis, especially confidence intervals, are better from a statistical standpoint of view? How does the costs for clicks, shipping and handling and discount affect the result and your recommendation and why? References: First & Last Name Assignment 3 – AB Testing applying Hypothesis Testing Table 1a: Click Percentages with Confidence Intervals Ad A BADM7020 Spring 2020 Figure 6: T-Test for Diff. of Purchase \% based on Clicks Table 1a: Click Percentages with Confidence Intervals Ad B Table 2: T-Test for Difference of Click Percentages Table 7: Average Revenues with 95\% CI based on Purchases Table 3: Purchase Percentages with 95\% CI based on Views Figure 8: T-Test for Diff. of Average Revenues based on Purchases. Figure 4: T-Test for Diff. of Purchase \% based on Views Table 9: Average Revenues with 95\% CI based on views Table 5: Purchase Percentage & 95\% CI based on Clicks Figure 11: T-Test for Difference of average Revenues based on Views First & Last Name Assignment 3 – AB Testing applying Hypothesis Testing BADM7020 Spring 2020 Table 12: Summary Table Ad A Type Click/View Purchase/view Purchase/Click Revenue/Purchase Revenue/View Revenue/view with discount Revenue/per view with Sh&H and discount Cost of clicks/view Net revenue/view Ad B Difference Mean LCL UCL Mean LCL UCL 19.28\% 17.69\% 20.96\% 26.30\% 24.51\% 28.16\% Diff 7\% LCL 4.5\% p for diff. 0.0001 Difference significant (Y/N)? Y Ad A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A View Click Ad 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 Puchase 0 0 0 0 0 1 0 0 0 1 0 0 1 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0 Revenue 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 506.3876 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 0 0 0 0 0 0 0 1 0 1 1 0 1 1 0 0 0 1 1 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 1 0 0 1 1 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 453.669 0 664.281 0 0 0 479.1344 0 0 0 0 0 0 0 0 0 0 0 437.4737 0 0 0 0 0 590.4246 0 0 0 0 0 0 560.3619 0 0 0 0 0 0 526.9507 0 A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 504.0676 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 462.4149 0 0 0 0 0 0 0 0 0 0 0 0 1 395.0707 0 0 0 0 0 0 0 0 0 0 0 0 A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 451.0959 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 482.312 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 1 0 0 1 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 611.1438 0 0 0 0 0 0 1 699.7409 0 0 0 0 0 0 0 0 ... Purchase answer to see full attachment