NAME

DESCRIPTION

For historical reasons, semantic concordance files are referred to as context files . Some of the field element within context files reflect that history, as well.

File Format

SGML attribute/value pairs are of the form:

       attribute = value

Our SGML format deviates from the standard in that value s are only enclosed in quotation marks when there may be embedded spaces. Due to the large number of attribute/value pairs, the presence of quotation marks around each value would have substantially increased the size of the concordances.

Nomenclature

File Structure

CONTEXTFILE ::= <contextfile concordance= conc >
       CONTEXT+
       </contextfile>

CONTEXT ::= <context filename= filename paras=yes>
       PARA+ | SENT+
       </context>

PARA ::= <p pnum= paragraph_number >
       SENT+
       </p>

SENT ::= <s snum= sentence_number >
       SENT_TOK+
       </s>

SENT_TOK ::= ( WORD_FORM | PUNC )+

WORD_FORM ::= <wf cmd=tag RDF SEP POS > word </wf>
       | <wf cmd=ignore DC SEP POS > word </wf>
       | <wf cmd=done RDF SEP POS SEM_TAG OT> word </wf>
       | <wf cmd=(update | retag) RDF SEP POS TAGNOTE NOTE> word </wf>

POS ::= pos= POS_TAG

POS_TAG ::= CC | CD | DT | EX | FW | IN | JJ | JJR | JJS | LS | MD | MD|VB
   | NN | NNP | NNPS| NNP|NP | NNP|VBN | NNS | NN|SYM | NP | NPS
   | PDT | POS| PP | PR | PRP | PRP$ | RB | RBR | RBS | RP
   | TO | UH | VB | VBD | VBG| VBN | VBP | VBZ | WDT | WP| WP$ | WRB

SEM_TAG ::= LEMMA WNSN LEXSN PN | NULL

LEMMA ::= lemma= lemma

WNSN ::= wnsn= sense_number

LEXSN ::= lexsn= lex_sense

PN ::= pn= CATEGORY | NULL

CATEGORY ::= person | location | group | other

RDF ::= rdf= redefinition | NULL

DC ::= dc= distance | NULL

SEP ::= sep=" separator_string " | NULL

TAGNOTE ::= tagnote= TAGNOTE_TYPE

TAGNOTE_TYPE ::= sns_miss | indist_sns | wd_miss | insuffctxt | sense_lost | misc

NOTE ::= note=" note "

OT ::= ot= OTHER_TAG | NULL

OTHER_TAG ::= notag | metaphor | idiom | complexprep | foreignword | nonceword

PUNC ::= <punc> PUNC_CHARACTER</punc>

PUNC_CHARACTER ::= [ , . ? ! , ; ( [ ) ] ` ' $ " : ]

Interpretation of SGML Elements

<contextfile concordance= conc >

This element indicates the beginning of a context file. conc specifies the name of the semantic concordance that the file is in. A semantic concordance file contains one or more context elements from the same semantic concordance. See semcor(7WN) for a list of semantic concordance names.

<context filename=filename paras=yes>

This element indicates the beginning of a context. filename is the name of the original corpora file from which this context was extracted. paras indicates that this document contains paragraph delimiters.

<p pnum=paragraph_number >

Start of a new paragraph. paragraph_number is an integer. The first paragraph in a context is numbered 1 , and paragraph numbers are incremented sequentially.

<s snum=sentence_number >

Start of a new sentence. sentence_number is an integer. The first sentence in each context is numbered 1 , and sentence numbers are incremented sequentially throughout the context . Sentence numbers do not restart at 1 in each paragraph.

<wf attribute/value_pairs > word </wf>

This element represents a word form. word is the orthographic form as it appears in the original document. All of the syntactic and semantic information is stored as attribute/value pairs described below.

cmd= cmd

Indicates the status of the wf element.

pos= pos

pos is the syntactic tag assigned by Eric Brill's stochastic part-of-speech tagger. See Syntactic Tags below for a list of possible values.

lemma= lemma

The base form of the word or collocation that the other attribute/value pairs in this wf pertain to. This is the form of the string used to search the WordNet database. If rdf is present, lemma is the base form of the redefinition. When pn is present, redefinition , lemma and category all have the same value.

wnsn= sense_number

sense_number is the integer sense number corresponding to the WordNet output display.

lexsn= lex_sense

lex_sense , when concatenated onto lemma using % as the concatenation character, creates a sense_key that indicates the WordNet sense to which word should be linked. This is the semantic tag for word . The format of a sense_key is described in senseidx(5WN) .

pn= category

Indicates that word is a proper noun categorized as one of the values of CATEGORY. When pn is present, redefinition , lemma and category all have the same value.

rdf= redefinition

If present, word has been "redefined" to something else. This is mainly used to define discontinuous collocations, correct typographical errors in the text, or to enter a string that should be used to search WordNet instead of word in order to find an appropriate sense for the semantic tag. When pn is present, redefinition , lemma and category all have the same value.

dc= distance

Indicates that word is part of a discontinuous collocation in which the words comprising the collocation are not adjacent. distance is an integer that specifies how many wf elements away the semantic tag for the collocation is. It may be either negative, indicating wf elements prior to this one, or positive, indicating wf elements following in the file.

sep=" separator_string "

Indicates that the space between this wf element and the next should be displayed as separator_string . The string may be one or more character. The default word separator is one space.

tagnote= tagnote_type

A tagnote attribute/value pair is always present if cmd is update or retag . This is used only during semantic concordance development, and indicates the type of problem encountered during semantic tagging. See TAGNOTE_TYPE for a list of possible values.

note=" note"

A note attribute/value pair is always present with tagnote . note may contain a string that provides additional information about the tagnote , or may be empty.

ot= other_tag

If present, a semantic tag cannot be assigned to word for one of the reasons listed in OTHER_TAG.

Syntactic Tags

SEE ALSO

Table of Contents

• NAME
• DESCRIPTION
• File Format
• Nomenclature
• File Structure
• Interpretation of SGML Elements
• Syntactic Tags
• SEE ALSO