CSCI 544 Homework 3

Error detection and correction: dealing with homophone confusion

Spring 2015
Kenji Sagae

Due date and time

Description

In written language, errors involving the use of words that sound similar or the same are fairly common. For example, the word its is frequently used where it's should, and vice-versa. Other confusable pairs include: they're/their/there, you're/your and loose/lose.

In this assignment you will develop an approach for detecting and correcting such errors. You only have to worry about these specific types of confusion (in either direction):

• it's vs its
• you're vs your
• they're vs their
• loose vs lose
• to vs too

You will be provided with one text file. You will need to create a new text file that differs only from the input file in that errors are corrected.

For example, given a file containing:

Then pour water or light oil from a graduated beaker into the chamber to fill the chamber too its gasket surface.
The horses moved at a clump; they were no more on parade than was they're driver; one fork of the road was as good as another.

you will turn in a new text file containing:

Then pour water or light oil from a graduated beaker into the chamber to fill the chamber to its gasket surface.
The horses moved at a clump; they were no more on parade than was their driver; one fork of the road was as good as another.

There may be zero, one or more corrections required per line.

Data, tools, etc.

Unlike in previous homework assignments, there is no further guidance on what approach to use, and no training data will be provided. Each student should be able to implement a suitable approach and find the appropriate data based on what has been covered so far in the course.

Also unlike in previous assignments, you may use open-source tools, libraries and toolkits (e.g. NLTK, Stanford CoreNLP). However, you may not use any commercial software or any free software for which source code is not provided.

Sample input and output files are available on Blackboard. These are meant to show only what input and output files look like; the test file (provided two days before the due date) may contain text of different genres, and errors are expected to be distributed differently. As a result, performance on the sample data may or may not match performance on the test set.

Turning in your assignment

Create a new bitbucket project called csci544-hw3 and give admin permissions to csci544-grader and to appliednlp. Do not add binary files, third-party software or data files (except for the output files, which of course must be included) in the project. Put the following in your repository:

• hw3.output.txt: the output text with corrections.
• README.md: a brief report that explains your approach, explains how to run the code, lists third-party software used, and provides any other information you find relevant to your submission.
• src directory: a directory called src that includes your code (scripts, programs, etc., but no binaries or third party software).

This assignment will be graded on a scale from 0 to 100 points, and will be worth 15% of your grade for the course.

Important information on collaboration and external resources

• This is an individual assignment. You may NOT work in teams or collaborate informally with other students in the class or anyone else. You must be the sole author of 100% of the code and writing you turn in.
• You may use free open-source third-party software, including NLTK and Stanford CoreNLP, but you MUST disclose the use of such software in your report.
• You may NOT use any commercial software, or any free software for which no source code is provided.
• You may NOT edit the output file manually.