WeSay News

If you aren’t yet using the new collaboration features of WeSay, you may have multiple versions of your dictionary out there.  Here are a few notes on ways to get them together.

The simplest case is where the users have been working on completely different sets of words, with no overlap. That is, they each started with completely empty dictionaries, which have never once been merged together.  In this specific case, you can merge them by hand.  Do that by opening each .lift file and copying all the <entry>…</entry> chunks of one file in next to the <entry>…</entry> chunks of the other file.    Open in WeSay to make sure you didn’t mess the lift file up.

In the more general case, you will want to merge them together using FieldWorks Language Explorer (FLEx).  To do that, follow these steps:

1) Create a new project using FLEx.

2) Import each .lift file into the project, one at a time, until you have a nice combined dictionary.

If getting/installing/using FLEx seems like to much, you can always just ask for someone to do this for you.  Write to the WeSay email list and ask someone to do the merge for you.

 

Updated 21 Jan, 2010: Changed instructions and screenshots to match the most recent release, which removes a few rough edges. We’re having a temporary problem with our Linux build, so these updates have not yet made it to the Linux release.

Since we first introduced WeSay a couple years ago, we’ve heard one request over and over: “Make it so that multiple people can collaborate on the dictionary”.  The need for this is actually greater than it was for ShoeBox, ToolBox, Lingualinks, or FieldWorks.  These other programs were primarily for linguists, and linguists working on smaller languages rarely have the luxury of a several partner linguists.  In contrast, it’s very likely that multiple members of the language community will want to contribute to a WeSay project.  In addition, WeSay is designed to be used in collaboration between an advisor and a native speaker.  They will most always have different computers and live in different places. All that to say we’ve known this was important for years, but it’s quite a challenge to get the collaboration powerful enough and yet simple enough for WeSay users.  Building this system, known as Chorus, has consumed the lion’s share of our new feature effort over the last year.

There’s another reason why we’ve been slow to put this feature out, and why we still (as of December 2009) are only slowly, cautiously introducing it.  It turns out that when we add a system which hooks all the team members together, we’ve just dramatically increased the potential for the entire team to suffer from one person’s mishap.  For example, imagine that one user deletes or renames a critically important file.  If that change is transmitted to the rest of the team, none of them will be able to keep working.  Some of these problems have proved hard to predict ahead of time.  As we identify such scenarios, we attempt to add protection against them.

We understand that people are tired of waiting.  And we could use more feedback to get this ready for Prime Time.  So I’m writing this blog for those of you who are willing to spend some time finding out if WeSay’s collaboration features are solid enough for your team.  If you find that they are, then great, you can start benefiting from them.  If you find that more work is needed, and you tell us about it, then we’ll be able to better prioritize our efforts to get you what you need.

Note: this blog entry is a work in progress… I expect to improve it over the coming weeks, as you try it out and give me feedback.

Eventually, getting started with WeSay and Language Depot needs to be a lot more simple than it is today. Please don’t hesitate to ask for further help.

Get yourself a LanguageDepot.org Account

Go to Language Depot and create yourself an account.

Please don’t use the same password you use for anything important… WeSay is NOT going to be careful about keeping your password well-hidden.

Get a LanguageDepot.org Project for the Language

Write to admin@languagedepot.org.  Please provide the following information:

• The name of the account you created in the previous step
• The name of the project.  Normally, the name of the language works well for this.
• The ISO 639-3 code for the language.  Easiest way to find that is to Google for “ethnologue thelanguagename”.

We will do three things:

1) Create the language project

2) Give you manager permissions on that project.  With those permissions, you will be able to assign additional contributors to the project, and turn features of the web site on and off.

3) Create a contributor account name “____Contributor” (with your code where the blanks are). 

People you have not added to the project will not be able to access your data.  However, we wouldn’t pretend to promise any real “security”.  If you need that, it’s perfectly ok to use a Mercurial server somewhere else… you aren’t tied to LanguageDepot.org.

Unless you tell us otherwise, we’ll assume it is ok for us to occasionally look at the files in your repository project for the purpose of fixing a problem for you or seeing how the collaboration features are being used in real projects.

Get the Data Together

It’s important that there is a single, up-to-date copy of the dictionary when you first put it up on Language Depot.  If there is currently only a  single person working on the dictionary, you need to get their project, and delete the project from their computer.  That does two good things: ensures they don’t keep working on it, and ensures that they will be using the proper version of the project later.

Make sure you backup the WeSay project from the stable version of WeSay you’ve been using (e.g. version 0.6).  If you decide that the development version (0.7) isn’t ready for you yet, you’ll want to go back to 0.6, and the files are not backwards-compatible.

If there are multiple copies of the dictionary out there, you need to do that for each one of them.  That is, get the project, remove it from their computer.  You have an extra step in this case, which is to merge the entries together.  Read these instructions on merging LIFT files.

Get the Most Recent Version of WeSay

The stuff shown here requires version 0.7 of WeSay, or greater.  Get the latest on the WeSay Downloads Page.

Install TortoiseHg

WeSay uses the Mercurial version control system to support sending data around and keeping a history of it.  For Windows, the easiest way to get Mercurial is from a free system named “TortoiseHG”, which you can download from here.

Push the project up to LanguageDepot

Ok, once you have a single dictionary folder with the whole team’s data, it’s time to do the initial push up to LanguageDepot.  At this point, WeSay can’t do this initial push for you.  The easiest way is to open a good ‘ol “command prompt” window.  If you don’t know how to do that, probably you’ve already enlisted a more technical person in previous steps. Get them to do this step too.

First, make sure Mercurial (hg) is properly installed. Type “hg version <return>”. You should see a little version message:

Now, change the directory to be that containing the WeSay dictionary.

Next, push the project up to Language Depot.  Here, instead of “wagi”, you’d type the ISO 639-3 language code you used when setting up the project.

You will be asked to enter the user name and password you used in setting up your Language Depot account.

Pull the project down to your computer

I recommend that you now pull the project right back to your computer as if you had never had it.  So if the project is already right where you want it, move it, rename it, or back it up and delete it.  Now, run the WeSay Configuration Tool, and click “Get From Internet”:

Next, we enter the account information:

And click Download.  The project will be downloaded to your computer. 

Begin Collaborating

You’ll notice a Send/Receive button now shows up on the dashboard:

Clicking it there will show a dialog like this:

Clicking “Internet” starts the synchronization:

Note: when WeSay detects that some changes were pulled down from the internet, it closes down and restarts itself so that it has a nice clean start with the new data.

Pull the project down to the computers of the rest of the team

Now do the same for each member of the team.  It’s fine to reuse the same “___Contributor” account for each of them… just make sure that their Windows/Linux account names are unique, as that is what will be used to keep track of who did what when you look at the project history.

“Advanced History”

Like most things in WeSay, you, the Advisor, need to turn on the collaboration features which are appropriate for your project.   There are two optional tasks you can enable if you want:

These show up on the dashboard under the “review” section:

The history show you all the changes the team has made:

It’s labeled “advanced history” because, frankly, this is more complex than what we expect many WeSay users to handle.  Use your own discretion. It may be that you, the advisor, will want this enabled in your configuration, but not in that of the rest of the team.

The Notes Browser

Collaboration is more than just sharing changes.  It turns out that as soon as you start working with others using Send/Receive, you find you want to write notes to them.  So we added the ability to attach questions to lexical entries, and to carry on conversations about the question.  Future versions will add other kinds of notes, and allow them to be attached to particular fields, not just whole entries.

In the following screenshot, notice the circled buttons.  We’d click the left-most to add a new question to this entry.  The other button represents a previous, unresolved question. A click on it brings up a dialog box in which we can read and answer the question.

Ok, but how do you find which entries have unresolved notes?  WeSay 0.7 also introduces the Notes Browser, which lets you find and interact with notes from all over the dictionary:

In addition to Questions, WeSay currently supports just one other kind of note, the Merge Conflict.  These are created by the automatic merger when two team members edit the same field at the same time. Unlike traditional version control systems, Chorus (the engine we’ve written to do all this) doesn’t stop the merge and force you to deal with the problem immediately.  Instead, it makes its best guess as to what to do, then creates a Merge Conflict Note which the team can read and deal with when it is ready.

If you don’t like what the merger did, go the entry and make whatever changes are necessary. Then click the “resolved” box to show that this has been dealt with. Or if you need to discuss what to do with a teammate, add a new message to the note. In the following screenshot, I’ve highlighted the hyperlink at the top, and the Resolved box at the bottom.

Ok, sorry, I know that’s a lot to absorb in one sitting. When we’re beyond this alpha testing period, I imagine I’ll reintroduce all of this pieces, with greater detail.

WeSay has always had Tasks which would show you just the words that needed some more information in a particular field.  However, the selection of which entries to show was pretty blunt:  if the field had an empty slot in any of its multiple writing systems, the task would show that entry.  This meant that you couldn’t easily set up WeSay for a user who, for example, just wanted to add vernacular definitions where English ones had already been entered. 

In another case, we might want to set a user up to add voice recordings of example sentences. But the task should only show example sentences where someone had previously entered in the example text.

The latest development release (0.5 build 2000) addresses this.  When you first create a project, tasks are configured to have the same behavior as before: an entry will be chosen if *any* of the writing systems assigned to that field are empty.

You can now limit the task to filling in the vernacular (gaw, in this example):

In addition, we can limit the task to only those entries where some other writing system has already been filled in:

Thanks, Mark, for taking the time to submit this request.  We would appreciate any feedback you can give us on this feature.  Does it work well for you?

The obvious next step would be to add a way make duplicates of some tasks, so that you could have both an “Add Examples” and an “Add Example Recordings” task.  This is now possible by editing the .wesayconfig file in a text editor.  If you want to know how, set me an message (hattonjohn at gmail).  That will tell me how much demand there is for it, and if there’s enough, we’ll make it easier.