[Planetlab-devel] moving from hrn to urn

[Josh Karlin]

Here is what I have at the moment:

. GIDs now only speak URN
. In methods that take a URN or HRN, the argument is referred to as an XRN
. Added an sfa/utils/xrn.py module for conversion functions
   . Conversion function takes (xrn,type=None) and returns (hrn,type)
. All methods/ and managers/ functions take an XRN instead of an HRN

Updating to HEAD will take me a few hours.  So I can do that and send 
the patch, or you can go ahead and submit your changes.  Whichever you'd 
prefer is fine with me.

Note, I had to flush my cache of certificates, delete 
/var/lib/sfa/authorities and do a clean import from plc to update the 
GIDs.  But I've made a script to test sfi.py and all of the operations 
work cleanly.

Josh



On 1/4/10 12:23 PM, Josh Karlin wrote:
> Sounds like we're clobbering each other.  I've already done the 
> conversion work.  Just need to merge my code with recent updates to 
> svn and can then send the patch.  Which route would be easier?
>
> Thanks,
>
> Josh
>
> On 1/4/10 12:19 PM, Tony Mack wrote:
>> Hi Josh,
>>
>> I've added 2 methods to the sfa.util.namespace module:
>>
>> http://svn.planet-lab.org/changeset/16433
>> sfa.util.namespace.hrn_to_urn()
>> sfa.util.namespace.urn_to_hrn()
>>
>> and I've updated sfa.methods.create_gid to convert the hrn field (now 
>> named hrn_or_urn) to parse this field and return:
>>
>> hrn, type # if the field is a urn
>> hrn, None # if the field is already a hrn
>>
>> I will go ahead and update the remaining methods if this looks ok to 
>> everyone.
>>
>>
>> Tony
>>
>>
>> ----- Original Message -----
>> From: "Josh Karlin"<jkarlin at bbn.com>
>> To: devel at lists.planet-lab.org
>> Sent: Wednesday, December 23, 2009 6:18:35 PM GMT -05:00 US/Canada 
>> Eastern
>> Subject: Re: [Planetlab-devel] moving from hrn to urn
>>
>>
>> Just wanted to be clear on this point. You're saying that registry 
>> interface methods will require URNs, while the other interfaces can 
>> take either an HRN or URN? I'm looking at methods/create_gid.py right 
>> now and trying to figure out what its parameters should be.
>>
>> Josh
>>
>>
>>
>> On 12/22/09 5:40 PM, Larry Peterson wrote:
>>
>> On the last point, the official registry calls all take a URN as
>> an argument. We have to fix all calling sites to translate the
>> HRN/Type pair into a URN before making the call.
>>
>> It also seems that we should change the GID implementation
>> to simply use the URN representation -- nothing extra has to
>> happen to get the right over-the-wire representation when
>> including the GID in some other structure. We then use the
>> get_hrn method to get the HRN we need for other stuff.
>>
>> Larry
>>
>>
>>
>> On Tue, Dec 22, 2009 at 3:41 PM, Josh Karlin<  jkarlin at bbn.com>  wrote:
>>
>>
>> This looks good to me. To make sure that I understand correctly there 
>> are two steps:
>>
>> 1) Alter the methods. Add (hrn=None, urn=None) to each method and 
>> then ensure that at least one of the two is provided. If a URN is 
>> provided, translate it to an HRN.
>>
>> 2) Adding a type to GID. Add the type to the GID class, and add 
>> get_urn(). It looks like a 'type' parameter needs to be added to 
>> trust/hierarchy.py's create_gid(). There is also the 
>> methods/create_gid.py method that needs to have a type parameter 
>> added. Are there other places that need to be edited?
>>
>> In order to get all over-the-wire calls to use URNs, what else would 
>> be necessary? For instance, methods like Resolve() take an HRN/URN 
>> and can call other registries, but currently it will make the call 
>> using an HRN instead of a URN.
>>
>> Josh
>>
>>
>>
>>
>>
>>
>> On 12/22/09 1:23 PM, Tony Mack wrote:
>>
>>
>> Scott you are correct. We can add 'type' to the GID and export a 
>> get_urn() method as well as the existing get_hrn(), making both 
>> formats available. Also, as Scott and Larry suggested, we can have 
>> our methods support both URN or HRN arguments and just convert the 
>> URN to HRN behind the scenes. Once this is done, we can support URN 
>> as the standard over-the-wire format, but still continue using HRN 
>> internally and on the command line.
>>
>> Tony
>>
>> ----- Original Message -----
>> From: "Scott Baker"<  smbaker at gmail.com>
>> To: "PlanetLab Development"<  devel at planet-lab.org>
>> Sent: Monday, December 21, 2009 6:29:18 PM GMT -05:00 US/Canada Eastern
>> Subject: Re: [Planetlab-devel] moving from hrn to urn
>>
>> My impression from discussions at the last GEC was that the only
>> substantial difference between the URN and HRN formats is that the URN
>> includes a type specifier for the final component of the URN. Both
>> formats are capable of supporting a multi-level hierarchy of
>> subauthorities. The rest of it is just syntax, and we could implement
>> a translation layer to translate one format into the other (with a few
>> subtleties, such as the potential need to escape out characters that
>> might be delimiters in one format, but aren't delimiters in the
>> other).
>>
>> The SFA GID is object-oriented. A logical first step in merging the
>> two naming schemes would probably be to add 'type' information to the
>> SFA GID. If type information was present, then the SFA GID object
>> could export a get_urn() method in addition to the get_hrn() method.
>> It's been a while since I worked with this part of the SFA code, so I
>> may be oversimplifying here. Tony will probably have the most up to
>> date opinion.
>>
>> There's also the reverse direction to consider, those SFA calls that
>> take an HRN as a parameter instead of a GID. Specifically I'm thinking
>> of something like Resolve. To convert URNs to HRNs here, all we need
>> to do is throw away the type information and rearrange the syntax.
>>
>> Our opinion at Raven is that we also prefer the simplicity of the HRN.
>> If we did make the change to support URNs internally, then we would be
>> interested in keeping the HRN format as a convenient shorthand for the
>> user.
>>
>> Scott
>>
>> On Mon, Dec 21, 2009 at 2:32 PM, Josh Karlin<  jkarlin at bbn.com>  wrote:
>>
>>
>>
>> According to the GMOC proposal there is room for sub-authorities:
>>
>> “urn:publicid:IDN+toplevelauthority[:sub-authority]*[\+resource-type]\
>> +resource-name”.
>>
>> So perhaps.. urn:publicid:IDN+plc:gpo+slice+mytestslice for
>> plc.gpo.mytestslice. ProtoGENI intends to support sub-authorities in the
>> future. As far as urn vs hrn, I have no preference. I would just like to
>> see the two SFA-based projects have compatible identifiers.
>>
>> Josh
>>
>>
>> On 12/21/09 4:35 PM, Larry Peterson wrote:
>>
>> A more substantive response...
>>
>> If we simply encode the HRN as the "resource name" in
>> the URN, then haven't we lost the multi-level nature of
>> the authority hierarchy. Specifically, the ProtoGENI web
>> page says:
>>
>> ------------------------
>> The definitive description of URN structure is given in the GMOC 
>> proposal.
>> However, as an informational summary, ProtoGENI URNs can be 
>> considered in
>> the form:
>>
>> urn:publicid:IDN+toplevelauthority+resource-type+resource-name
>>
>> where:
>>
>> toplevelauthorityis an internationali[sz]ed domain name (which must 
>> match
>> the one in the certificate of the authority which issued the object 
>> name)
>> resource-typeis a string describing the type of the named object (the 
>> set of
>> strings is described below) resource-nameshould uniquely identify the 
>> object
>> among any other resources with identical toplevelauthority and
>> resource-type. It is important to realise that the ProtoGENI API 
>> attaches no
>> other significance to this field, and in particular, no relation is 
>> implied
>> between objects with identical resource-name but differing 
>> toplevelauthority
>> or resource-type. However, individual authorities (and especially 
>> component
>> managers) may choose to define additional semantics for resource names.
>> ------------------------But for SFA, the authority that issued the
>> certificate might be plc.eu.inria,
>> not plc (aka planet-lab.org ). An alternative is to put plc.eu.inria 
>> in the
>> toplevelauthority field, and let the name be the last "token" in the 
>> HRN.
>>
>> I guess this boils down to knowing the right way to use URNs when you
>> have a chain of authorities rather than a simple 2-level hierarchy.
>>
>> Larry
>>
>> On Mon, Dec 21, 2009 at 4:17 PM, Larry Peterson<  pete.larry at gmail.com>
>> wrote:
>>
>>
>>
>> Well, I didn't exactly agree to replace HRNs with URNs...
>>
>> I do think there's general agreement among PlanetLab
>> developers that UUIDs can go away, but (1) I'm personally
>> not yet convinced that trading HRNs for URNs is the right
>> thing to do, and (2) there are quite a few others with a stake
>> in the SFA that need a chance to weigh in.
>>
>> I would like to retain the simpleness of HRNs in the sfi. Maybe
>> that can be done with a simple transformation, but even if that's
>> the case, I'd like to hear the as to argument as to why we need
>> type information encoded in the name (since it's already recorded
>> in the registry record).
>>
>> So let the comments begin...
>>
>> Larry
>>
>> On Mon, Dec 21, 2009 at 3:52 PM, Josh Karlin<  jkarlin at bbn.com>  wrote:
>>
>>
>>
>> Greetings,
>>
>> After recent discussion between the GENI Project Office, Larry, and Rob,
>> it has been agreed that some (hopefully minor) alterations should be 
>> made to
>> the SFA and ProtoGENI to make them more compatible. The GPO has offered
>> development assistance. Christopher Small was on the project but he has
>> recently moved to NetApp. I am taking over for Chris and am looking 
>> at the
>> first step, which is to implement a common global identifier. Larry has
>> agreed to use ProtoGENI's URN names for SFA objects, but I don't 
>> believe a
>> formal description has been agreed upon.
>>
>> The URN format ( http://www.protogeni.net/trac/protogeni/wiki/URNs ) is
>> of the form:
>>
>> urn:publicid:IDN+toplevelauthority+resource-type+resource-name
>>
>> types include: authority, interface, link, node, slice, sliver, ticket,
>> and user.
>>
>> For instance, in ProtoGENI you might have
>> 'urn:publicid:IDN+ emulab.net +slice+mytestslice'. For PlanetLab's 
>> SFA, I
>> imagine we might have
>> 'urn:publicid:IDN+ planet-lab.org +slice+plc.gpo.mytestslice' such 
>> that the
>> old hrn is placed in the 'resource-name' field. The only real difference
>> from the current hrn is that the identifier is longer and the type is
>> included. It would be good to have some discussion on this.
>>
>>
>> Regardless of the selected format, it will take a fair bit of work to
>> make the appropriate changes to the SFA code. I have started a list of
>> design decisions that need to be made:
>>
>> 1) Should URNs be stored natively in the geni records or should they be
>> produced only when interfacing with ProtoGENI nodes? The idea is to move
>> the SFA over to URNs completely so it should be in the records in my
>> opinion.
>>
>> 2) Should the geni record change format? E.g. remove the redundant
>> 'type' field. For simplicity I recommend not changing the format at 
>> first
>> (since a lot of code appears to use the type field) and removing it 
>> later if
>> necessary.
>>
>> 3) Should sfi.py use the old hrn or the new urn? I think Larry would
>> prefer the user to deal with the simpler hrn format at the command 
>> line. So
>> the sfi.py script would have to convert old hrns to the new urns.
>>
>> 4) Should the term 'hrn' be replaced with 'urn' in the code? Not sure if
>> it matters much.
>>
>> 5) How will SFA Tables interact with the new URN? It seems like it will
>> have to process the top-level-domain as well as the object name.
>>
>> I hope to begin working on this soon, so please add to the discussion 
>> and
>> voice your concerns and comments.
>>
>> Thanks,
>>
>> Josh
>>
>>
>> _______________________________________________
>> Devel mailing list
>> Devel at lists.planet-lab.org
>> https://lists.planet-lab.org/mailman/listinfo/devel
>>
>>
>>
>> _______________________________________________
>> Devel mailing list
>> Devel at lists.planet-lab.org
>> https://lists.planet-lab.org/mailman/listinfo/devel
>>
>>
>> _______________________________________________
>> Devel mailing list
>> Devel at lists.planet-lab.org
>> https://lists.planet-lab.org/mailman/listinfo/devel
>>
>>
>>
>> _______________________________________________
>> Devel mailing list
>> Devel at lists.planet-lab.org
>> https://lists.planet-lab.org/mailman/listinfo/devel
>>
>> _______________________________________________
>> Devel mailing list
>> Devel at lists.planet-lab.org
>> https://lists.planet-lab.org/mailman/listinfo/devel
>>
>>
>> _______________________________________________
>> Devel mailing list
>> Devel at lists.planet-lab.org
>> https://lists.planet-lab.org/mailman/listinfo/devel
>>
>> _______________________________________________
>> Devel mailing list Devel at lists.planet-lab.org 
>> https://lists.planet-lab.org/mailman/listinfo/devel
>> _______________________________________________
>> Devel mailing list
>> Devel at lists.planet-lab.org
>> https://lists.planet-lab.org/mailman/listinfo/devel
>>
>> _______________________________________________
>> Devel mailing list
>> Devel at lists.planet-lab.org
>> https://lists.planet-lab.org/mailman/listinfo/devel
>
> _______________________________________________
> Devel mailing list
> Devel at lists.planet-lab.org
> https://lists.planet-lab.org/mailman/listinfo/devel