Page 1 of 6 123456 LastLast
Results 1 to 10 of 60

Thread: xAP Weather Schema Schema add change request

  1. #1
    Automated Home Jr Member pete_c's Avatar
    Join Date
    Feb 2011
    Location
    Near Chicago IL, USA
    Posts
    39

    Default xAP Weather Schema Schema add change request

    I am curious what I need to do to request a schema change in the weather report class?

    Today its:

    weather.report
    {
    UTC=(Time of report in hh:mm format using utc time zone)
    -- mandatory
    DATE=(Date of report in YYYYMMDD format)
    -- mandatory
    WindM=(Value of wind in mph or "Gusty" if gusty)
    -- optional
    WindK=(Value of wind in kph or "Gusty" if gusty)
    -- optional
    WindGustsM=(Value of wind gusts in mph)
    -- optional
    WindGustsK=(Value of wind gusts in kph)
    -- optional
    WindDirC=(Compas heading of wind N|NE|E|SE|S|SW|W|NW)
    -- optional
    WindDirD=(Compas heading of wind in degrees)
    -- optional
    TempC=(Temperature in centigrade)
    -- optional
    TempF=(Temperature in fahrenheit)
    -- optional
    DewC=(Dew point in fahrenheit)
    -- optional
    DewF=(Dew point in fahrenheit)
    -- optional
    AirPressure=(Air pressure in hPa)
    -- optional
    Cloud=(Overal cloud cover currently only "Clear Skies")
    -- optional
    CloudM.X=(Cloud type and height in miles ie "Overcast at 5.4M. X increments on each cloud layer)
    -- optional
    CloudK.X=(Cloud type and heightin kilometers ie "Overcast at 5.4Km. X increments on each cloud layer)
    -- optional
    Icon=(The name of the picture to use, icon names from kweather www.kde.org. File names below)
    -- optional
    Error=(Download errors either "NoData" if last connection failed or "StationNotFound" if ICAO code isn't valid)
    -- optional
    }
    Steve; the author of Cumulus added Rain (in mm I believe) which I use. It would be better though to change the Rain part to:

    RainM=x.yy
    RainI=x.yy

    and another ideal add would be relative humidity:

    Humidity=xxx.yy

    I am currently playing with the weather.report schema with Cumulus testing the Fine Offset and Davis Vantage Vue weather stations.

    I've asked the author of a Linux based similiar to Cumulus application to add XaP / weather.report schema or if I could help add XaP to his application.

    I'm looking to move the connection(s) over to a sheevaplug currently testing the Linux application on a Seagate Dockstar.
    Last edited by pete_c; 7th February 2012 at 05:14 PM.

  2. #2
    Automated Home Sr Member kev_t's Avatar
    Join Date
    Jun 2004
    Location
    Lincolnshire
    Posts
    58

    Default

    Hi Pete

    A couple of things to note about the Weather Schema:-

    It is not a xAPWeather Schema, ie it is not a formal core xAP Schema with a specification like for instance xAPBSC

    If you look at the schema, most of the keys are optional. Also you will see that most data 'types' eg wind, are available in both imperial and metric forms

    This schema originated many years ago and I think was based on the data either from a one wire weather station or the data aquired from a page scrape, or perhaps both.

    I can think of 3 readily availbale sources of Weather Schema data

    mi4's weather connector, which grabs metar data like
    2012/02/08 10:20
    EGNJ 081020Z 00000KT 9999 BKN026 00/M02 Q1044

    The WeatherDisplay program which interfaces with a variety of weather stations

    Cumulus which similarly interfaces with a variety of weather stations

    The implementation is based on the schema, but which options are selected depends on the inclinations of the implementor and the data available. In addition other keys may also be added. So the Weather Display program produces messages like

    {
    v=12
    hop=1
    uid=FF400200
    class=Weather.Report
    source=windy.weather.1
    }
    Weather.Report
    {
    tempf=34.2
    utc=10:52:25
    tempc=1.2
    icon=Sunny
    date=20120208
    windm= 0
    windk= 0
    airpressure=1041.1
    winddirc=N
    humidity=88
    rain=0.0
    extratemp1=-100.0
    extratemp2=-100.0
    extratemp3=-100.0
    extrahum1=-100
    extrahum2=-100
    extrahum3=-100
    }

    So you can see here that he has added rain (in inches) and humidity, but left out for instance wind direction in degrees (regrettably).

    The 'extra...' keys come from someone elses extention to the schema, I think for one-wire, which have been slavishly copied here but are not used.

    James' (mi4) xAPWeather connector produces messages like

    xap-header
    {
    v=12
    hop=1
    uid=FF400100
    class=Weather.Report
    source=mi4.weather.egnj
    }
    Weather.Report
    {
    UTC=10:20
    Date=20120208
    WindM=Gusty
    WindK=Gusty
    TempC=0.0
    TempF=32.0
    DewC=-2.0
    DewF=28.4
    AirPressure=1044
    Icon=cloudy3
    CloudK.0=Broken cloud at 0.8km
    CloudM.0=Broken cloud at 0.5M
    }
    Time
    {
    TOD=Day
    Time=11:06:55
    SunRise=07:29
    SunSet=16:58
    }

    Which just decodes the METAR data and adds an extra section for Sunrise and Sunset based on Latitude and Longditude. Regrettably since the BBC has changed their Weather site and / or the location indexes the non metar data sources no longer work and so the Weather Forecast messages are not available

    I don't have examples of the Cumulus message to hand, perhaps you could post an example here?

    For completeness, there is another, very different, version of the schema on Patrick Lidstones site, but this does not seem to be the common usage, or adopted as the schema on the official xAP site and so probaly should be avoided.

    So to answer your main question, from the above, I would advise that you base your extension to the schema on what has gone before

    I have a spare Dockstar here so I would be very interested to hear if you make any progress with that

    I am sure there may be other comments, but you can see above what has gone before

    Hope this helps

    Kevint
    (The other kevin)

  3. #3
    Automated Home Jr Member pete_c's Avatar
    Join Date
    Feb 2011
    Location
    Near Chicago IL, USA
    Posts
    39

    Default

    Hello Kevint,

    Understood.

    Here too now many of the online weather services are looking/changed to charge a monthly service fee to access their data and starting not to provide weather data for "free" anymore. I've gone now to doing screen scrapes and snapshots of HTML pages for weather maps and such.

    Steve has mentioned that he added rain to the Cumulus XaP broadcast; but it wasn't the standard and is hestitate to add "undocumented" XaP features to his application.

    Its like the "which came first" scenario; the chicken or the egg.

    I've used the AAG (Dallas designed anemometer) for a bit over 10 years or so along with a Dallas 1-wire Rain guage. Great stuff; 1-wire; whole home is covered here both inside and out with 1-wire devices.

    Just recently started to "test" the Fine Offset and Dallas Vantage Vue. Have the two of them gathering data and comparing it to the AAG. The AAG has always over reported wind speeds; think that's a debounce design flaw thing though. (I have 2 AAG wind speed devices which over report wind speeds).

    Here is the Cumulus data from the Fine Offset and the Davis Vantage Vue:

    xap-header
    {
    v=12
    hop=1
    uid=FF437500
    class=weather.report
    source=sanday.cumulus.ICS-HST
    }
    weather.report
    {
    UTC=13:47
    DATE=20120208
    WindM=8.0
    WindK=12.9
    WindGustsM=13.0
    WindGustsK=20.9
    WindDirD=293
    WindDirC=WNW
    TempC=-2.6
    TempF=27.4
    DewC=-5.7
    DewF=21.7
    AirPressure=1029.2
    Rain=0.0
    }
    xap-header
    {
    v=12
    hop=1
    uid=FF437500
    class=weather.report
    source=sanday.cumulus.ICS-STORAGE-2
    }
    weather.report
    {
    UTC=13:48
    DATE=20120208
    WindM=3.8
    WindK=6.1
    WindGustsM=11.4
    WindGustsK=18.4
    WindDirD=315
    WindDirC=NW
    TempC=-2.6
    TempF=27.3
    DewC=-6.2
    DewF=20.8
    AirPressure=1031.4
    Rain=0.0
    }
    Today I have two Seagates running. The one in "production" is running a sprinkler control program connected to two Rain8Nets via USB. It has a web server / web interface and a Touchscreen interface running on ArchLinux. Talks and receives XaP.

    The second Seagate in test mode is running "Dockstar Debian" and a linux (similiar to Cumulus) program called "weather view" plus a 1-wire read one wire application (which is broadcasting XaP). It is connected to both the Davis Vantage view (serially ) and the Midon Temp0x serially (via USB to serial devices).

    The author of Cumulus doesn't want to add anything XaP which is not documented; for fear he is not standardized I guess.

    Thank you KevinT.

    Attached are pics of my Seagate setup.
    Attached Images Attached Images
    Last edited by pete_c; 8th February 2012 at 03:12 PM.

  4. #4
    Automated Home Sr Member kev_t's Avatar
    Join Date
    Jun 2004
    Location
    Lincolnshire
    Posts
    58

    Default

    To be clear,

    There is no specification to adhere to for the Weather.Report schema

    There is a description of the schema, much of which is optional

    The data in the schema should be based on the data available from the data source

    Keys which are optional need not be included

    And, most importantly, where additional data is available then it should be added, since applications which parse the message should gracefully ignore anything they don't understand.

    Steve of Cumulus/Sanday fame should therefore not be concerned about adding extra keys, The most obvious of which is humidity, which is already available in the message from WeatherDisplay.

    I know there have been comments on the ommision of Humidty from the Cumulus weather message in other forums, eg that for the Livebox HAH.

    Thanks for posting the sample messages and updating us on what you are doing

    (and pretty please can we call it xAP rather than any other variation of upper and lower case? )

    kevint

  5. #5
    Moderator Kevin's Avatar
    Join Date
    Jan 2004
    Location
    West Yorkshire
    Posts
    558

    Default Weather Schema additions

    Can I just weigh in with a view somewhere inbetween.

    The weather schema is not an 'official' xAP schema in that it doesn't have a schema name starting xAP... however it is presented as a standard template in that if everyone adheres to it then it doesn't matter which weather station or software you use they will operate the same (acknowledging that some sensors may not be available on some stations). This is a fundamental tenet in xAP.

    Therefore it is important that things are done in a unified way . It's no good if developers are adding Hum= Hum%= RelHum= RHum= HumP= if you get my drift. Where units are involved even more problematic, the RainM might imply miles if you look at CloudM for example and RainM has an implied period of measurement.

    So I would like to try and get some concensus on what needs to be added and then get those included on the website within a schema revision and everyone benefits from a standard approach.

    We did revisit the schema before, looking to make it complete but it's very easy to get bogged down with units and measurement periods for example rain in the last hour, day, month, year etc. Let's try and get the fundamanental missing bits added, and maybe take an approach of adding additional blocks for period based data.

    You are of course free to include your own paramaters into xAP message blocks* although I've never really been comfortable with this - especially in terms of using common parameter names as they can later conflict with schema revisions. It was really intended as a 'for your own purposes' thing rather than for a released application. Regardless, extra parameters should not break anything.

    So Pete and others , suggest away, I'm keen to get something better out there too. What I would ask is to give it some good thought re parameter names, units, values and completeness so that we try and make it as thorough and capable as possible.

    K

    * edit: Just wanted to add that I agree that xAP developers should add extra keys to a schema when they've deficiencies or have extra data to include - but really with a purpose of suggesting such keys become part of a schema revision so we all benefit. Humidty is a good example here. Yes , you can add your own keys for your own purposes but that may not be appropriate for a released application and if that significantly changes the whole schema then probably it should be in another block or a new schema. There's no reason you can't support two schema concurrently either.
    Last edited by Kevin; 9th February 2012 at 02:07 AM.

  6. #6
    Automated Home Sr Member kev_t's Avatar
    Join Date
    Jun 2004
    Location
    Lincolnshire
    Posts
    58

    Default Consensus

    I had thought that by publishing some examples of the current implementations, that have been in use for some time that we had demonstrated a consensus

    What we are talking about here is adding two keys, rain and humidity which are pretty fundamental to a weather report

    And in any case any receiving application should gracefully ignore, ie not break, if it gets keys it does not recognise


    kevint
    Last edited by kev_t; 9th February 2012 at 10:14 AM.

  7. #7
    Moderator Kevin's Avatar
    Join Date
    Jan 2004
    Location
    West Yorkshire
    Posts
    558

    Default

    Kevin,

    So .. for example... what does

    Rain= show ?

    K

  8. #8
    Automated Home Jr Member pete_c's Avatar
    Join Date
    Feb 2011
    Location
    Near Chicago IL, USA
    Posts
    39

    Default

    Yes; relating the the current schema; I am and have used 1-wire instruments for many years. I am "testing" today with three weather stations. The xAP data on the xAP network is a real hodgepodge of data.

    This said I am attempting to convey a message (s) of an agreed upon standard for some addicted home automation enthusists and a shift to utilizing
    xAP as a means of home automation transport. (a do all of sorts).

    That said the author of Cumulus is aware of the currently documented weather report xAP schema and has told me (written) that he is hesitant to make changes to his application as he doesn't want to step beyond the boundaries of said documentation or "step on any toes".

    My "testing" went to utilizing very common "other" weather stations; IE: Davis and the Fine Offset and attempting to utilize broadcast xAP data with the common Cumulus application. A whole different subject but currently benchmarking the three devices using just xAP data graphing numbers such that folks can see in vivo what xAP can do for them.
    Last edited by pete_c; 9th February 2012 at 02:52 PM.

  9. #9
    Automated Home Sr Member kev_t's Avatar
    Join Date
    Jun 2004
    Location
    Lincolnshire
    Posts
    58

    Default

    KH

    I guess you are referring to the fact that the rain= key does not specify the units

    We already have at least two implentations which just report rain=

    Pete says his Cumulus is reporting in mm
    My WD is reporting in inches

    From a quick look at each program this is configurable, and you use the units with which you are most familiar or approprioate to your localisation.

    I don't have a problem with the key not specifying units as I expect my rainfall to be in inches since that is how I configured WD, and I would probably notice if it misreported an inch of rain as 25.4

    A discussion on fixing this schema took place in 2008 and was not resolved and has been raised on a number of different support forum since, we should progress this now and update the wiki so we don't have to revisit

    kevint
    Last edited by kev_t; 9th February 2012 at 02:44 PM.

  10. #10
    Automated Home Jr Member pete_c's Avatar
    Join Date
    Feb 2011
    Location
    Near Chicago IL, USA
    Posts
    39

    Default

    That would be great!

    Personally if the numbers don't give me what I want; then I just convert them; IE: mm to in or in to mm. I do the conversions "on the fly" as the data is being entered into a database.

    But that is me and my setup.

    An xAP "newbee" would assume or will just utilize the numbers "as is"; plug n play like.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •