IS4800 – Empirical Research Methods in Information Science

 

 

[Syllabus] [Schedule] [Homework] [Projects] [Bibliography] [Resources] [Directory] [Grads]

 

Homework (Individual Assignments)

 

The following assignments are to be completed individually and emailed to is4800@ccs.neu.edu by noon on the due date. All homeworks should be emailed as a single Word or PDF file, with relevant R output (graphs, tabular output, statistics, but not commands) copied and pasted into the document, along with any other materials considered for grading (e.g., survey instrument).

I1

Individual Homework #1 (not graded - due next class)

 

  1. If you do not have a CCIS account and have not requested one, you should do so immediately. An account is required to access the online readings. See the Resources page for details.
  2. Send an email with your name and preferred email address to is4800@ccs.neu.edu.

  

 

I2a

Individual Homework #2a - Ethnography (due 1/30)

 

First, read these two papers to get a better understanding of the role of Ethnography in Information Science:

                              

Northeastern has just hired you to make Snell library more efficient and friendly. Pick a location where talking is acceptable and spend an hour people watching with a notebook and pencil. Spend some time watching the kinds of activities that people are engaged in, and think about how technology could help improve these activities. Pick one such activity to focus on and study. For your chosen activity watch several people perform the task and make detailed notes about the series of steps they go through, any objects ("artifacts") they use, whether they interact with other people and, if so, the step-by-step details of this interaction. Conduct two or more unstructured interviews with your subjects.

 

Some examples of activities (you can study one of these or, better yet, come up with your own):

  • finding a meeting room
  • finding a book
  • ordering at the cafe
  • rendezvousing with friends

 

What to Email  Your report should include a one-paragraph summary of why you picked your particular activity to focus on, followed by an overview of the activity, the kinds of people you observed engaging in it, and a description of any artifacts they used. Describe the individuals you interviewed (not by name) and what you learned from the interviews. Following this, provide a detailed description of the activity and any variations you observed. Grading will be based largely on the coherence and organization of your report, the level of detail in your observations, and your analysis and interpretation. Here's a good example from a prior class.

 

I2b

Individual Homework #2b - Designing Research Models (due 1/30)  

 

Identify two measurable variables from your ethnographic study that might conceivably be associated. Design separate descriptive, correlational and experimental research designs for studying these variables. For each, include a diagram (such as the one in Figure 6. from the sample research plan, but with just one or two boxes, as appropriate) and text describing the purpose of the study, the measures you plan to use, and what the results would be useful for. Identify a possible "third variable" that might invalidate predictions made with results from the correlational study and how this will be controlled in the experimental study.

 

Optional R exercise: Pick at least 5 of your favorite albums. Create an Excel spreadsheet with the following columns: Album, Tracks, Cost, PctCost (with the last column reflecting the cost of each album as the percent of the cost of all albums in this collection). Fill in the data. Save a copy to csv format and import to R. Create a bar chart showing the number of tracks per album. Create a pie chart showing PctCost labeled by Album. Include these charts in your report.

 

 

I3

Individual Homework #3 - Usability Testing (due 2/6)

 

First, read through the following papers as examples of usability/performance testing of a user interface (in this case new sensing inputs for a PDA):

 

Next, pick an obscure piece of software with a user interface (ideally one you may have created for a class). Define two simple tasks using the software (something you can describe in 1-3 sentences and take less than 3 minutes to do) and write them down on two pieces of paper. Select two or more interval or ratio measures from pages 194-195 of the Nielsen reading that you think may be relevant to the software, in addition to at least one nominal or ordinal measure (could be sociodemographic).

 

Ask three (or more) classmates or friends to help you with a user study. Make sure they have not used the software before. Obtain verbal consent (as discussed on 1/19). Provide a brief description of the software (but not how to use it). Then, give each participant each task and watch them attempt to complete it. Do not provide any help. Collect your measures.

 

Submit a brief writeup of your test plan, descriptive statistics of your data, and any design recommendations resulting from your tests.

Here is a sample writeup of a descriptive analysis.

 

You may work individually or in teams of two. 

 

I4

Individual Homework #4 - Designing a Composite Measure for a Questionnaire (due 2/16)

 

Design a new composite self-report measure to assess a person's Homework Procrastinaton (degree to which they procrastinate on their homework). Assume it only has one factor, but use at least five scale items. Incorporate information from at least one literature reference. Assess the face and content validity of your measure and work through a bivariate analysis of your items.

 

Decide on one method for assessing validity (besides face & content) for your measure that you can also assess in your questionnaire. This should be an additional question (or an additional previously validated composite measure) on your survey and should provide a numeric measure.

 

Add any other items to your questionnaire that you feel are important. Implement your questionnaire on surveymonkey, Google forms, or similar.

 

Email your questionnaire to the class (is4800-all@ccs.neu.edu). (You are also obligated to reply to any questionnaires mailed to you within 48 hrs.)

 

Collect your data.

 

Compute the reliability (internal consistency) of your measure using R Alpha. Compute descriptive statistics for your measure and any other items you may have included on the questionnaire. Assess the validity of your measure (you can do this qualitatively, e.g., using scatterplots).

 

Document and submit all of the above.

 

You may work individually or in teams of two. 

 

I5

Individual Homework #5 - Designing and Analyzing an Experiment (due 2/27)

 

Work individually on this one.

 

Part 1 - Designing an Experiment

Write a research plan for conducting an experiment comparing WizziWord vs. CoolText word processors using admins from BigBucks, Inc (both of these are new products). Outcome measures to include productivity (words per day output during the 8th week after the new word processors are introduced), and satisfaction, using the ILoveWordProcessors 12-item index (Cronbach alpha=0.82, test-retest correlation of 0.93, correlation with the standard 100-item WordProcessorsAreGreat index was 0.72). From studies at other sites you expect to see a difference in productivity of approximately 3,000 (SD 1,200) words per day between the products. Assume a 50% response rate to your recruitment ad, and a 85% retention rate for subjects. Be sure to include the following in your plan:

  • Hypotheses
  • Research model (the boxes and arrows diagram) and description of variables/measures
  • Human subjects issues, including eligibility criteria, recruitment procedures, and the number of potential subjects you need to reach with your recruitment ad.
  • Detailed protocol , including recruitment, sampling and randomization methods
  • Analysis plan

Refer to sample research plan for inspiration. Your complete plan should be about 2-3 pages long, single spaced.

 

Part 2 - Analyzing Data from an Experiment Using R

You've successfully completed your study, and have the following data. Write a 1-2 pages Results and Discussion section of study report describing your findings. (Hint: Be sure to review the normality of data, but for this exercise you can assume all distributions are normal.)

 

 

ID Condition Performance Satisfaction
1 C 5293 3.1
2 C 1602 2.4
3 C 6231 2.3
4 C 2350 1.4
5 W 4873 4.4
6 W 4429 6.2
7 W 2322 2.4
8 W 4239 5.2
9 C 3214 2.3
10 C 2431 4.9
11 W 3214 5.9
12 W 5321 6.1
13 C 4276 3.4
14 C 2302 3.9
15 W 4712 4.8
16 W 6904 6.2