The Open Linguistics Working Group
Status Quo and Perspectives, by Christian Chiarcos and Sebastian Hellmann
Since its formation last year, the Open Linguistics Working Group (OWLG) has been steadily growing and the direction the working group is heading has been clarified (although a number of issues remain open). In the last months, we concentrated on the identification of goals and directions for this working group to pursue, and in this blog post, we summarize results of this process, about its current status as well as the main challenges and problems we have identified so far.
An important result of our discussion are the seven points described in the next section, which define the purpose of the working group. In the next section, we summarize four major problems and challenges of the work with linguistic data. Such problems will become a primary topic of the Working Group. Thereafter, we give an overview of the current status and activities of the group and provide some suggestions for how to get involved.
PurposeAs a result of numerous discussions with interested linguists, NLP engineers and information technology experts, we identified seven open problems for our respective communities and their ways to use, to access and to share linguistic data. These represent the challenges to be addresses by the working group, and the role that it is going to fulfil:
- Promote the idea and definition, as specified in opendefinition.org of open data in linguistics and in relation to language data.
- Act as a central point of reference and support for people interested in open linguistic data.
- Provide guidance on legal issues surrounding linguistic data to the community.
- Build an index of indexes of open linguistic data sources and tools and link existing resources.
- Facilitate communication between existing groups.
- Serve as a mediator between providers and users of of technical infrastructure.
- Assemble best-practice guidelines and use cases to create, use and distribute data.
Open linguistics resources, problems and challengesAmong the broad range of problems associated with linguistic resources, we identified four major classes of problems and challenges during our discussions that may be addressed by the OWLG. First, there is a great uncertainty with respect to legal questions of the creation and distribution of linguistic data; second, there are technical problems such as the choice of tools, representation formats and metadata standards for different types of linguistic annotation; third, we have not yet identified a point of reference for existing open linguistic resources; finally, there is the agitation challenge, i.e., how (and whether) we should convince our collaborators to release their data under open licenses. These challenges are described below in detail. 1. Legal questions The linguistic community becomes increasingly aware of the potentially difficult legal status of different types of linguistic resources:
- How to find a suitable license for my corpus ?
- Whose copyright do I have to respect ? For example, corpora may have complex copyright situations where the original authors own the primary data, and thus may have partial copyright on the entire collection.
- Are there exceptions (e.g. for academic research) to the copyright that may allow me to work with my corpus anyway ?
- How to circumvent (or solve) copyright issues ?
- What legal restrictions apply to a particular resource (e.g., web corpora, newspaper corpora, digitizations of printed editions, audio and video files) ?
- How to create multi-media (audio, video) data collections in a way that allows us to use (and hopefully, distribute) them for research ?
Current status and on-going developments (as of May, 19th, 2011)So far, we focused on the task to delineate what questions the Open Linguistics Working Group may address, to formulate its general goals and potentially fruitful application scenarios. This blog entry summarizes these discussions, and it concludes a critical step in the formation process of the working group: Having defined a (preliminary) set of goals and principles, we can now concentrate on the tasks at hand, and in to collect resources and to attract interested people in order to address the challenges identified above. At the moment, our Working Group assembles 32 people from 21 different organizations and 7 countries (Germany, US, UK, France, Canada, Hungary, and Slovenia). Our group is relatively small, but continuously growing and sufficiently heterogeneous. It includes people from library science, typology, historical linguistics, cognitive science, computational linguistics, and information technology, just to name a few, so, the ground for fruitful interdisciplinary discussions has been laid out. We are very glad that famous linguists such as Nancy Ide (Text Encoding Initiative, American National Corpus, Vassar College) and Christiane Fellbaum (WordNet, University of Princeton) accepted our invitation to post guest blogs, and we would like to intensify this tradition and encourage all members of the OWLG to describe interesting projects and experiences on this medium, to share insights and difficulties over the Open Linguistics mailing list, and, of course, to join our meetings and telcos. The next meeting is about to be held in conjunction with the Fifth Open Knowledge Conference (OKCon), June 30th to July 1st 2011 in Berlin, Germany, and of course the OKCon itself is a great reason to join us there. As for our first concrete activities, we have begun to compile a list of resources of particular interest to the members of the working group. Most of these resources are free, others are partially free (i.e., annotations free, but text under copyright), and a few have been included that are very representative for a particular type of resource (e.g., corpora derived from the Penn Treebank as a prototypical multi-layer corpus). Altogether, the list comprises 102 entries by now, and the next step would be to register them at the CKAN metadata repository and to select a few for deeper investigation. One aspect of such investigations may be the conversion of some of the resources to RDF and to provide them as Linked Data. Several working group members (including the authors of this blog) are working towards this direction. The ultimate result may be an Linguistics Linked (Open) Data cloud, as sketched in the graphic to the right (click to enlarge). On this basis, novel applications in all participating fields may be developed.
Get involvedHaving all that said, we hope to have encouraged others to contribute and to join. And if indeed we succeeded in doing so, you may be interested in how to join and how to contribute:
How to join
- Register your (open) resources at CKAN (and please, don’t forget to tag them as “linguistics”)
- Attend meetings / telcos (announced over the mailing list
- Write blog posts for our blog
- Become a group administrator on CKAN (on request)