Authoring VOEvent packets

In [1]:
from __future__ import print_function
import voeventparse as vp
import datetime

(To get started reading VOEvents, see the previous notebook).

Packet creation

We’ll start by creating the skeleton of our VOEvent packet. We set the role to test so that nobody is tempted to start acting on the contents of this demo event. We also set the timestamp in the Who block to the time the event was generated (not when the observation was made), as per the specification:

In [2]:
v = vp.Voevent(stream='hotwired.org/gaia_demo', stream_id=1,
                       role=vp.definitions.roles.test)
In [3]:
#Set Who.Date timestamp to date of packet-generation:
vp.set_who(v, date=datetime.datetime.utcnow(),
        author_ivorn="foo.hotwired.hotwireduniverse.org/bar")
vp.set_author(v, title="Hotwired VOEvent Hands-on",
                      contactName="Joe Bloggs")
v.Description = "This is not an official Gaia data product."

At any time, you can use vp.dumps (dump-string) to take a look at the VOEvent you’ve composed so far:

In [4]:
# print(vp.dumps(v, pretty_print=True))

However, that’s pretty dense! Use vp.prettystr to view a single element, which is a bit easier on the eyes:

In [5]:
print(vp.prettystr(v.Who))
<Who>
  <Description>VOEvent created with voevent-parse, version 1.0.3+4.g58fc1eb. See https://github.com/timstaley/voevent-parse for details.</Description>
  <AuthorIVORN>ivo://foo.hotwired.hotwireduniverse.org/bar</AuthorIVORN>
  <Date>2018-09-29T16:10:28</Date>
  <Author>
    <title>Hotwired VOEvent Hands-on</title>
    <contactName>Joe Bloggs</contactName>
  </Author>
</Who>

Adding What content

We’ll add details from this GAIA event:

Name UTC timestam p RA Dec Aler tMag His tMa g HistS tdDev Cla ss Comment Publish ed
Gaia 14ad i 2014-11- 07 01:05:09 168. 4784 1 -23. 0122 1 18.7 7 19. 62 0.07 unk now n Fading source on top of 2MASS Galaxy (offset from bulge) 2 Dec 2014, 13:55

Now let’s add details of the observation itself. We’ll record both the magnitude that Gaia is reporting for this particular event, and the historic values they also provide:

In [6]:
v.What.append(vp.Param(name="mag", value=18.77, ucd="phot.mag"))
h_m = vp.Param(name="hist_mag", value=19.62, ucd="phot.mag")
h_s = vp.Param(name="hist_scatter", value=0.07, ucd="phot.mag")
v.What.append(vp.Group(params=[h_m, h_s], name="historic"))

Adding WhereWhen details

Now we need to specify where and when the observation was made. Rather than trying to specify a position for Gaia, we’ll just call it out by name. Note that Gaia don’t provide errors on the position they cite, so we’re rather optimistically using 0:

In [7]:
import pytz
vp.add_where_when(v,
               coords=vp.Position2D(ra=168.47841, dec=-23.01221, err=0, units='deg',
                                    system=vp.definitions.sky_coord_system.utc_fk5_geo),
               obs_time=datetime.datetime(2014, 11, 7, 1, 5, 9, tzinfo=pytz.UTC),
               observatory_location="Gaia")
In [8]:
## See how much element creation that routine just saved us(!):
print(vp.prettystr(v.WhereWhen))
<WhereWhen>
  <ObsDataLocation>
    <ObservatoryLocation id="Gaia"/>
    <ObservationLocation>
      <AstroCoordSystem id="UTC-FK5-GEO"/>
      <AstroCoords coord_system_id="UTC-FK5-GEO">
        <Time unit="s">
          <TimeInstant>
            <ISOTime>2014-11-07T01:05:09</ISOTime>
          </TimeInstant>
        </Time>
        <Position2D unit="deg">
          <Name1>RA</Name1>
          <Name2>Dec</Name2>
          <Value2>
            <C1>168.47841</C1>
            <C2>-23.01221</C2>
          </Value2>
          <Error2Radius>0</Error2Radius>
        </Position2D>
      </AstroCoords>
    </ObservationLocation>
  </ObsDataLocation>
</WhereWhen>

Adding the How

We should also describe how this transient was detected, and refer to the name that Gaia have assigned it. Note that we can provide multiple descriptions (and/or references) here:

In [9]:
vp.add_how(v, descriptions=['Scraped from the Gaia website',
                                        'This is Gaia14adi'],
                       references=vp.Reference("http://gsaweb.ast.cam.ac.uk/alerts/"))

And finally, Why

Finally, we can provide some information about why this even might be scientifically interesting. Gaia haven’t provided a classification, but we can at least incorporate the textual description:

In [10]:
vp.add_why(v)
v.Why.Description = "Fading source on top of 2MASS Galaxy (offset from bulge)"

Check and save

Finally - and importantly, as discussed in the VOEvent notes - let’s make sure that this event is really valid according to our schema:

In [11]:
vp.valid_as_v2_0(v)
Out[11]:
True

Great! We can now save it to disk:

In [12]:
with open('my_gaia.xml', 'wb') as f:
                vp.dump(v, f)

And we’re all done. You can open the file in your favourite text editor to see what we’ve produced, but note that it probably won’t be particularly elegantly formatted - an alternative option is to open it in your browser.

Advanced Usage

Free-style element authoring

Note that if you want to do something that’s not part of the standard use-cases addressed by voevent-parse, you can always use the underlying lxml.objectify tools to manipulate elements yourself. For example - don’t like the ‘voevent-parse’ tag that gets added to your VOEvent Who skeleton? You can delete it:

In [13]:
## Before deletion:
## (Enclosed in an if-clause in case this is re-run after the cell below)
if hasattr(v.Who, 'Description'):
    print(v.Who.Description)
VOEvent created with voevent-parse, version 1.0.3+4.g58fc1eb. See https://github.com/timstaley/voevent-parse for details.
In [14]:
if hasattr(v.Who, 'Description'):
    del v.Who.Description
#Now it's gone!
print(vp.prettystr(v.Who))
<Who>
  <AuthorIVORN>ivo://foo.hotwired.hotwireduniverse.org/bar</AuthorIVORN>
  <Date>2018-09-29T16:10:28</Date>
  <Author>
    <title>Hotwired VOEvent Hands-on</title>
    <contactName>Joe Bloggs</contactName>
  </Author>
</Who>

Want to add some additional elements of your own? Here’s how, but: make sure you stick to the VOEvent schema!

In [15]:
import lxml.objectify as objectify
In [16]:
# This won't last long:
vp.valid_as_v2_0(v)
Out[16]:
True

If you just want a single text-value element (with no siblings of the same name), you can take a syntactic shortcut and simply assign to it:

(Under the hood this assigns a new child ``StringElement`` to that tag - if there are pre-existing elements with that same tag, it is assigned to the first position in the list, overwriting anything already there.)

In [17]:
v.What.shortcut='some text assigned in a quick-and-dirty fashion'
print (vp.prettystr(v.What))
<What>
  <Param dataType="float" name="mag" ucd="phot.mag" value="18.77"/>
  <Group name="historic">
    <Param dataType="float" name="hist_mag" ucd="phot.mag" value="19.62"/>
    <Param dataType="float" name="hist_scatter" ucd="phot.mag" value="0.07"/>
  </Group>
  <shortcut>some text assigned in a quick-and-dirty fashion</shortcut>
</What>

In general, though, you probably want to use SubElement, as this allows you to create multiple sibling-child elements of the same name, etc.

In [18]:
for i in range(5):
    objectify.SubElement(v.What, 'foo')
    v.What.foo[-1]="foo{}".format(i)
print("I have {} foos for you:".format(len(v.What.foo)))
print (vp.prettystr(v.What))
I have 5 foos for you:
<What>
  <Param dataType="float" name="mag" ucd="phot.mag" value="18.77"/>
  <Group name="historic">
    <Param dataType="float" name="hist_mag" ucd="phot.mag" value="19.62"/>
    <Param dataType="float" name="hist_scatter" ucd="phot.mag" value="0.07"/>
  </Group>
  <shortcut>some text assigned in a quick-and-dirty fashion</shortcut>
  <foo>foo0</foo>
  <foo>foo1</foo>
  <foo>foo2</foo>
  <foo>foo3</foo>
  <foo>foo4</foo>
</What>

In [19]:
# Get rid of all the foo:
del v.What.foo[:]

Alternatively, you can create elements independently, then append them to a parent (remember how lxml.objectify pretends elements are lists?) - this is occasionally useful if you want to write a function that returns an element, e.g. to create a new Param (but voevent-parse wraps that up for you already):

In [20]:
temp = objectify.Element('NonSchemaCompliantParam', attrib={'somekey':'somevalue'})
v.What.append(temp)
print(vp.prettystr(v.What))
<What>
  <Param dataType="float" name="mag" ucd="phot.mag" value="18.77"/>
  <Group name="historic">
    <Param dataType="float" name="hist_mag" ucd="phot.mag" value="19.62"/>
    <Param dataType="float" name="hist_scatter" ucd="phot.mag" value="0.07"/>
  </Group>
  <shortcut>some text assigned in a quick-and-dirty fashion</shortcut>
  <NonSchemaCompliantParam somekey="somevalue"/>
</What>

Obviously, these are non-schema compliant elements. Don’t make up your own format - use Params for storing general data:

In [21]:
vp.valid_as_v2_0(v)
Out[21]:
False

Note that you can get a traceback if you need to figure out why a VOEvent is non-schema-compliant - this will report the first invalid element it comes across:

In [22]:
try:
    vp.assert_valid_as_v2_0(v)
except Exception as e:
    print(e)
Element 'shortcut': This element is not expected.