Category Archives: Technical Issues

Password reset fixed

One of the side effects of our server move was the need to reconfigure our locally-hosted transactional email services, which were always a little flaky — if you have tried to reset your password lately you will no doubt have noticed that the promised email never arrived.

That’s fixed now. We  switched from self-hosted to using Postmark, which works very well and should be far more stable.

Server move in progress

…from Rackspace to Digital Ocean and from one large-ish server to several smaller ones, from a relatively prehistoric version of RedHat to the latest Ubuntu 14LTS, from Zend Server+Apache to nginx (openresty actually), and from PHP5.2 to PHP5.5. It really wasn’t as painful as it sounds like it should have been. As part of the move, we also made some improvements in error and uptime reporting, integrated into a Slack channel for near real-time monitoring.

Actually the move is pretty much complete, although we’re still tinkering a bit with the caching and load balancing setup — we apologize for any brief outages you may experience while we do that — since we couldn’t afford to completely replicate the entire server stack in a staging environment.

Getting browser/proxy/server caching right is actually taking more time than we anticipated. We’ll let you know here as soon as we turn our attention to other things. In the meantime, if you see something, please say something — we’ve got issues!

Server issues

Normally the Registry just hums along quietly and doesn’t demand too much attention. But the last system update we performed seems to have altered our memory usage pretty dramatically and we’re quite suddenly having out-of-memory issues and some heavy swapping. We’ve expanded the server capacity twice already as a stopgap while we investigate, but before we move to an even larger server we’re testing some alternative configurations.

In the meantime there may continue to be periodic slowdowns, but you should see some improvement in a few days.

The last thing we want is for you to think we’re not seeing, or ignoring, the problem.

Whoops

Until lately we’ve been pretty happy with our ISP, Dreamhost. But a few months ago, after several during-the-presentation meltdowns of the Registry we determined that we needed to move to a higher-performing, more reliable server. We could have done the easy thing and moved to a Virtual Private Server at Dreamhost. Instead, we setup an entirely fresh server in the Rackspace Cloud and very carefully, with much testing created a fresh instance of the Registry with greatly expanded data capacity, some updated code, and considerably more speed. So far, so good.

We had a self-imposed deadline of several weeks before the DCMI 2011 Conference in The Hague and completely missed it. This left us with the choice of waiting until after the conference to redirect our domain to the new server or taking the risky step of switching domains just a few days before the conference. Of course, we didn’t wait. At which point we discovered that we couldn’t simply redirect our main domain to the new server but needed to redirect the subdomains as well, breaking our wiki and blog. Which we had a great deal of difficulty restoring while on the road to DCMI.

But everything’s back to normal now, and even updated. We now resume our regular programming.

SKOS updated for Vocabularies

Just a quick note that today we updated the version of SKOS that we provide for describing value vocabularies. This deprecates the properties that were removed from the final SKOS release and adds the many new ones. We’ve also restricted the non-mapping relation properties (skos:broader, skos:narrower, skos:related) to the ‘containing’ scheme while providing cross-scheme mapping for the mapping relations.

We don’t yet provide a useful interface for building collections, but that’s coming real soon now.

Oh, and we added a SPARQL endpoint.

Registry Installation Instructions

Jeepers, no posts for 3+ months and then two in one day! The truth is that I hadn’t realized the last post was still sitting in my drafts folder more than a month after I wrote it.

Moving on…

A number of folks have been interested in installing the Registry, especially since we’ve talked before about ‘easy installation’ being one of our design goals.

We’re pleased to announce that we have finally tweaked things to make a reasonably simple install from our subversion repository possible and provided some hopefully simple instructions detailing how to get the Registry up and running. We don’t provide enough tweaking or instructions (yet) to fully customize the interface, so once it’s installed it’ll still look exactly like the Registry, just running on your server instead of ours.

Whenever we update the production server, we’ll tag that code in subversion and update the link in the instructions (tying a string around my finger to help me remember as we speak), but there won’t be any other ‘release’ announcement unless we do something major.

Whenever we modify the database structure, we’ll provide a sql script to alter the database with each release. These scripts will always modify the database as it was after the previous release, so if you skip releases you’ll need to run the scripts sequentially. But this will all be on the instructions page.

We expect to update the production code quite often over the next few months.

Metadata Schema

If you’ve been watching the Registry closely (and we know you have), you’ll have noticed that a few weeks ago we started supporting the registration of metadata schemas. It’s not finished and far from perfect, but the perfect can often be the enemy of the good and at the moment it’s, well, good enough for now.

What makes it tough to get schema registration right is that our approach to what we’re calling registration attempts to be cross-cultural — trying to create a bridge from the technologies supporting the Semantic Web to the somewhat more ‘traditional’ data transfer technologies like XML.

We’re also trying to ‘eat our own dog food’ and are using an internally registered Application Profile to define the properties we’re using to describe metadata schemas and ultimately Application Profiles. This AP helps drive the schema registration user interface and we hope at some point we’ll be able to use a registered AP to generate many different interfaces, both human and application. It’s arguably too ambitious, but baby steps…

Vocabulary Management
The Registry is really more Vocabulary Management Application than Registry at this point, since we’ve layered so many management services on top of the basic registry functions. It manages two types of vocabularies:

  • Value vocabularies — unordered lists of values (terms) that we express as skos concept schemes in RDF and a simple enumeration in XML Schema
  • Class/Property vocabularies — lists of classes, properties (or attributes depending on your mental model) that we currently express as rdf:properties and rdfs:classes

Much of our terminology (value vocabularies, metadata schema, application profile) stems from our work with the Dublin Core Community more than the Semantic Web Community and maybe we’ll refactor some of those names as we move forward. But we hope the semweb folks can translate and we hope that the DC folks won’t hold our ultimate departure from some of their terms against us.

In the meantime, feel free to play in the sandbox.

TimeSlices and Versions

You can now retrieve a snapshot in time of the RDF or XSD serialization of a Concept/Scheme/Vocabulary by appending a ‘TimeSlice’ to the URI. For example: http://metadataregistry.org/uri/NSDLEdLvl/ts/20060422200002.rdf or http://metadataregistry.org/uri/NSDLEdLvl/ts/20060422200002.xsd will always and forever retrieve the SKOS/RDF or XML Schema representation of the NSDL Ed Level Vocabulary as it appeared at 2 seconds after 8PM on April 22, 2006 (2006-04-22 20:00:02). If you follow the above .rdf link you’ll notice that the concept URIs that the TimeSliced Vocabulary references have also had a TimeSlice appended: http://metadataregistry.org/uri/NSDLEdLvl/1001/ts/20060422200002.rdf in order to lock them into that precise point in time on an individual basis as well. We hope the utility of being able to reference a vocabulary at a particular point in time regardless of subsequent changes will be, well, useful.In order to make retrieving TimeSlices for specific events in the history of a vocabulary a bit handier, we added a TimeSlice link to every history event. You can specify a TimeSlice for any point in time regardless of its relationship to a history event, but the link just makes it simpler (it’s over on the right side of each line):

The Registry! :: NSDL Education Level Vocabulary :: History of ChangesUploaded with plasq‘s Skitch!

Named Versions

You’ll maybe also have noticed that there’s a ‘Name’ link nestled to the right of the RDF and XSD links. If you’re a Vocabulary Administrator, then you now have the ability to label a TimeSlice with a distinct version name. That link again is there to make it easy to reference a point in historical time and clicking on it pre-enters that TimeSlice into the Create new version form:

The Registry! :: Creating new versionUploaded with plasq‘s Skitch!There’s no limit to the number of versions you can create, and versions (unlike TimeSlices) can be deleted or edited by any Vocabulary Admin:

The Registry! :: NSDL Education Level Vocabulary :: List VersionsUploaded with plasq‘s Skitch!…although we think that either editing or deleting a version is likely to be a less-than-ideal practice. Still, we allow it — it’s your vocabulary after all.Once a named version has been associated with a TimeSlice, it will appear in the event history list just above the point in time it references:

The Registry! :: NSDL Education Level Vocabulary :: History of ChangesUploaded with plasq‘s Skitch!The RDF and XSD links on the right side of the version line now reference the version name:http://metadataregistry.org/uri/NSDLEdLvl/version/release+1.0.rdfBut this is where it gets a little incorrect… Since the named version URL is just a TimeSlice reference, it does a silent redirect to the referenced TimeSlice. It should probably do a 303 redirect instead. We’ll fix this later, unless it’s a show-stopper for one of our many users.

Improved user management

We’ve been promising for a while now that we’d make it easier, actually a better word is ‘possible’, for Vocabulary Owners to add ‘Members’ to Owner/Agents and ‘Maintainers’ to Vocabularies. We finally implemented it today! It’s unfortunate that it has taken us so long, since one of the primary goals of the Registry is to support multi-user vocabulary development, but it turned out to require more infrastructure twiddling than we thought it would.

If you’re a vocabulary owner and are logged in, you can add other registered folks as ‘members’ of your Owner/Agent and you can even make them administrators if you want:

The Registry! :: NSDL Registry :: List Members

The Registry! :: Editing NSDL Registry permissions for Diane Hillmann

Uploaded with plasq‘s Skitch!

We hope the process is pretty self-explanatory.

Once you’ve added a user as a member of your Owner/Agent group, you can add them to your vocabularies as Vocabulary Maintainers or Administrators.

The Registry! :: NSDL Registry Agents Vocabulary :: List Maintainers

The Registry! :: Adding maintainer to NSDL Registry Agents Vocabulary
Uploaded with plasq‘s Skitch!

We realize that this is still somewhat limited and of course the documentation is ummm, poor, but we’ll be doing more with user management shortly.