[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: [Syslog-sec] RE: protocol-11.txt - sysUpTime
Hi all,
please let me elaborate a little on my experience with syslog use cases.
As far as I have seen, there are two major use cases:
#1 trouble shooting/setup
This is an activity done close to real-time. Here, the raw syslog
messages are reviewed by the administrator. The most important part in
this use case is the free-form message. It tells the administrator which
problems the device is experiencing or other status data that is
valuable at that very instant. Common scenarios are setting up or
troubleshooting router filter settings or firewall rule sets. Here, the
device is telling via syslog which rules fired and which ones not. So
the administrator can check his setup. Also, this use case often applies
if a new application/device/whatever is being setup in a network and the
admin checks messages from the new device and/or existing devices -
especially if it doesn't work as it should. This use case might be
called the "tail -f /var/log/messages" case - just to stress my point...
The common thing about this scenario is that the administrator reviews
raw message in more or less real time.
#2 reporting/analysis
In this use case, consolidated syslog data is viewed. Raw data is NOT
viewed. Typically, the analysis is not real-time but acting on past data
(though there are some analysis tools which work in real-time or
close-to-real-time and then issue alerts based on the result of their
ongoing analysis). In this use case, a program/script/whatever
programmatically reads the syslog entries and somehow transforms them
into a format that is more understandable by a human. The administrator
views the (transformed) end result.
Of course, these use cases do not exclude each other. For example, it
might very well be that an administrator detects an anomaly in a
consolidated report (use case #2) and then investigates its cause via
real-time trouble shooting (use case #1). Eventually, he will generate
other reports on the fly before he looks at the raw data. Eventually
he'll never look at any raw data because it is not required for that
troubleshooting purpose.
Obviously, there are a lot of different use cases. But based on my
experience the two above apply to the majority of cases.
Now if I look at what the administrator actually does, I would expect
that in use case #1 sysUpTime will be of little to no help. The reason
is that the administrator either has recently set up / rebooted the
device and so he knows how long it is up. Or he might have come from a
transformed report which included the information. In any case, I think
sysUpTime will not be the admins prime focus. In fact, I'd say that
he'll probably ignore STRUCTURED-DATA at all and just focus on the
human-readble part of the message (MSG).
In case #2, sysUpTime will be helpful, but primarily to the report
logic. I assume that a decent reporting engine will convert any
user-unfriendly data type into information that the admin can easily
make sense of.
Again, there are more use cases than the two outlined above. Also, I do
not claim to have absolute knowledge about what users do with their
syslogs. However, I am working for a long time in this field and this is
honestly what I have seen in practice.
If (and only if) my observations are true, that would mean we could use
the "user-unfriendly" SNMP integer and only need to specify that it must
be represented in string form (because syslog does not support binary
fields).
Any feedback is highly appreciated.
Rainer
> -----Original Message-----
> From: syslog-sec-bounces@www.employees.org
> [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of
> David B Harrington
> Sent: Saturday, April 23, 2005 2:28 PM
> To: 'Tom Petch'; 'syslog'
> Subject: RE: [Syslog-sec] RE: protocol-11.txt - sysUpTime
>
> If the expectation is that syslog messages will be read in syslog raw
> format by humans more than by programs, then I agree it should be done
> in a presentation format in the message. Under those conditions, I
> suggest breaking out the days, hours, minutes, etc.
>
> If the expectation is that this field will be used by prgrams more
> than humans, then the raw format woul dbe better.
>
> Of course, if there are suitable use-cases for each, you could also
> achieve both a human-friendly and a machine-friendly approach:
> "142days:2hr:8min:13sec:9 (123456789)".
>
> David Harrington
> dbharrington@comcast.net
>
> > -----Original Message-----
> > From: Tom Petch [mailto:nwnetworks@dial.pipex.com]
> > Sent: Saturday, April 23, 2005 6:03 AM
> > To: ietfdbh@comcast.net; syslog
> > Subject: Re: [Syslog-sec] RE: protocol-11.txt - sysUpTime
> >
> > David
> >
> > Yes, your wish to clarify the intended users of syslog
> > messages is best done
> > first. I assume it is humans like myself (even though I have
> > coded log
> > analysers).
> >
> > Having clarified the users, then I will press again for a
> > defined presentation
> > format. In syslog, sysUpTime is an SD-ID so it is encoded in
> > UTF8 and as far as
> > I know, UCS only has the concept of characters, not of
> > integers, so if the
> > presentation format is
> > to be standardised, we must do it.
> >
> > Tom Petch
> > ----- Original Message -----
> > From: "David B Harrington" <ietfdbh@comcast.net>
> > To: "'Tom Petch'" <nwnetworks@dial.pipex.com>; "'Rainer Gerhards'"
> > <rgerhards@hq.adiscon.com>; <syslog-sec@employees.org>
> > Sent: Thursday, April 21, 2005 11:48 PM
> > Subject: RE: [Syslog-sec] RE: protocol-11.txt - sysUpTime
> >
> >
> > > Hi Tom,
> > >
> > > Thank you for the good desacription of your concerns.
> > > You are correct; RFC3418 does not have a DISPLAY-HINT.
> > >
> > > SNMP is designed to be a programmatic interface, not a
> > human-friendly
> > > interface; SNMP management applications often convert the
> unfriendly
> > > sysUpTime into days: hours: minutes: seconds, and so on, as you
> > > suggest.
> > >
> > > Syslog is designed to be more user-friendly. Syslog certainly can
> > > convert the sysUpTime value into something more readily
> > understood by
> > > humans. One concern I have with that approach is that, in the SNMP
> > > world, we try to keep all unnecessary processing out of the
> > agent and
> > > in the management application, so the device being managed can
> focus
> > > on forwarding packets or whatever, and not on converting
> > raw data into
> > > friendlier forms. While minimizing the management processing of
> > > internetworking devices was critically important in 1980,
> > and is stil
> > > important in resource-limited devices such as set-top boxes, many
> > > systems now have at least some capability to spare.
> > >
> > > So my question is, do the bulk of syslog users read the raw syslog
> > > messages without any automation, or do the bulk of operators now
> use
> > > tools to help filter syslogs, given the tremendous amount of
> > > information available in the logs? I'm sure there are multiple
> > > use-cases. [I do not know the answer to the question; it is not
> > > rhetorical.]
> > >
> > > If they are already using tools to view (and possibly correlate)
> the
> > > messages, could those tools do the conversions of the raw
> sysUptime,
> > > or is it really necessary for the originator to do the conversion
> of
> > > sysuptime when logging the message?
> > >
> > > Do operators view the raw messages under certain circumstances,
> such
> > > as when they are troubleshooting in real-time? Are they
> > likely to also
> > > be looking at the raw SNMP as well, say as an Ethereal
> > packet capture
> > > of both syslog and SNMP messages? If so, then having the syslng
> raw
> > > sysUptime match the SNMP raw sysUptime might be useful; if they
> are
> > > comparing syslog and SNMP traffic, and SNMP gives them an
> > integer, and
> > > syslog gives them a formatted string in
> > > days:hours:minutes:seconds:hundredths format, that would make it
> > > harder for them rather than easier.
> > >
> > > This WG seems to be working toward improving the ability to
> > correlate
> > > syslog and SNMP events. What are the use-cases that this WG
> > is trying
> > > to target with this effort at consistency between syslog and SNMP?
> > > What are we trying to achieve by having the SNMP sysUptime in the
> > > syslog message?
> > >
> > > Is there a real goal here, or is consistency with SNMP just
> > > feature-creep?
> > >
> > > David Harrington
> > > dbharrington@comcast.net
> > >
> > > > -----Original Message-----
> > > > From: syslog-sec-bounces@www.employees.org
> > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf
> > Of Tom Petch
> > > > Sent: Thursday, April 21, 2005 4:12 PM
> > > > To: Rainer Gerhards; syslog-sec@employees.org
> > > > Subject: Re: [Syslog-sec] RE: protocol-11.txt - sysUpTime
> > > >
> > > > I will try again.
> > > >
> > > > I believe we need a specification of how sysUpTime is
> > > > displayed in the syslog
> > > > message and I do not believe the I-D gives it. ( I am happy
> > > > to use the concept
> > > > of sysUpTime as defined in RFC 3418).
> > > >
> > > > The I-D refers to syntax and I see a problem here.When SNMP,
> > > > as in RFC3418,
> > > > says it has a syntax of TimeTicks by this it means a data
> > > > type, an object type,
> > > > in this case a integer of up to 32 bit (which can be encoded
> > > > in one or two or
> > > > three or four octet) and not much more. Elsewhere in the
> > > > IETF I find syntax
> > > > used in a different, wider sense. So I was, and I am sorry I
> > > > did not spell it
> > > > out more, suggesting that using syntax in the context of
> > > > syslog when referring
> > > > to SNMP was likely to lead to confusion. In which sense is
> > > > the word being used?
> > > >
> > > > Some definitions in SNMP have display hints associated with
> > > > them; as far as I
> > > > know, there is none for sysUpTime (ie TimeTicks) - David will
> > > > put me right if I
> > > > am wrong - and I struggle to think of a useful one. 378 is
> > > > fine when I am
> > > > analysing the log of the early stages of a reboot; 1576800000
> > > > is not very
> > > > friendly when the server has been up for six months. (And
> > > > even if there is a
> > > > display hint somewhere in tthe RFC, you might need the help
> > > > of a MIB doctor to
> > > > find it:-).
> > > >
> > > > So I see the reference to RFC 3418 as leaving the field wide
> > > > open for any
> > > > representation of a time interval which could be as large as
> > > > 2**32 - 1 /100
> > > > second or as small as 0.01. I know of nothing in SNMP to
> > > > stop it being
> > > > represented in days:hours:minutes:sec.onds. And this feels
> > > > right; sysUpTime is
> > > > not a user-friendly concept, rather something that a low cost
> > > > agent can cheaply
> > > > handle; different SNMP managers present it differently (and
> > > > yes, some do it in
> > > > days etc)..
> > > >
> > > > I think we should nail the representation down. I do not have
> > > > a good suggestion
> > > > for what that should be. Probably ddddddd.hh in seconds;
> > > > having an integer in
> > > > units of hundredths of seconds is more architecturally pure
> > > > but will confuse
> > > > those not familiar with SNMP, which I suspect will be the
> > > > majority of syslog
> > > > users.
> > > >
> > > > Tom Petch
> > > >
> > > > ----- Original Message -----
> > > > From: "Rainer Gerhards" <rgerhards@hq.adiscon.com>
> > > > To: <syslog-sec@employees.org>
> > > > Sent: Thursday, April 21, 2005 8:59 PM
> > > > Subject: [Syslog-sec] RE: protocol-11.txt - sysUpTime
> > > >
> > > >
> > > > David,
> > > >
> > > > I did specify this based on Tom's comment that the SNMP
> > > > definition could
> > > > not be used for syslog. I reviewed the SNMP RFCs once again
> > > > and thought
> > > > the point was proved. As it looks, that was wrong. So I will
> > > > revert back
> > > > to the previous definition which simply states that it
> > should be RFC
> > > > 3418 compliant. Thanks for pointing this out.
> > > >
> > > > Rainer
> > > >
> > > > > -----Original Message-----
> > > > > From: David B Harrington [mailto:ietfdbh@comcast.net]
> > > > > Sent: Thursday, April 21, 2005 7:51 PM
> > > > > To: Rainer Gerhards; 'syslog'
> > > > > Subject: protocol-11.txt - sysUpTime
> > > > >
> > > > > Hi,
> > > > >
> > > > > You state that semantics and syntax are as defined in RFC
> 3418,
> > > then
> > > > > proceed to define a different syntax. It is a bad practice to
> > > claim
> > > > > consistency with something, and then reinvent it but
> > keep calling
> > > it
> > > > > the same thing. We should decide what we want here.
> > > > >
> > > > > If the goal is consistency with SNMP, then we should use the
> > > syntax
> > > > > used for the SNMPv2-MIB sysUpTime [RFC 3418]. That syntax is a
> > > > > TimeTicks value (INTEGER 0..4294967295 hundredths of a
> > > > second, with no
> > > > > decimal points) since reinitialization of the (SNMP)
> management
> > > > > system.
> > > > >
> > > > > If the goal is to define a syslog-specific version of
> > > > sysUpTime, then
> > > > > we should skip the reference to RFC 3418, call it something
> > > > else, and
> > > > > define it fully as a syslog field. If we go this route,
> > > > then we should
> > > > > discuss what re-initialization of the management system
> > means - is
> > > > > this re-initialization of the SNMP management system,
> > or is it the
> > > > > re-initialization of (a particular part of) the syslog
> > system? We
> > > > > might also want to document rollover behavior.
> > > > >
> > > > > David Harrington
> > > > > dbharrington@comcast.net
> > > > >
> > > > > > ####
> > > > > > 7.3.2 sysUpTime
> > > > > >
> > > > > > The "sysUpTime" parameter MAY be used to include the SNMP
> > > > > > "sysUpTime"
> > > > > > parameter in the message. Its syntax and semantics are
> as
> > > > > > defined in
> > > > > > RFC 3418 [12].
> > > > > >
> > > > > > In syslog, it is represented as a decimal string with a
> > > maximum
> > > > > of
> > > > > > two digits for fractional seconds. Full seconds and
> > > fractional
> > > > > > seconds MUST be delimited by a period ("."). Leading
> > > > > > zeros MUST NOT
> > > > > > be used for full seconds. For example, a
> > "sysUpTime" of one
> > > > > minute
> > > > > > MAY be represented as "60", "60.0", or "60.00", but
> > > > not as "060"
> > > > > or
> > > > > > "60.000".
> > > > > > ####
> > > > > >
> > > > > > I am not so proficient with SNMP, but I think (as you said)
> > > > > > TimeTicks is
> > > > > > actually integer. So we should have a maximum value
> > defined plus
> > > a
> > > > > > rollover behaviour. Or does this mean we also need to
> include
> > > > > > an epoch?
> > > > > > ;)
> > > > > >
> > > > > > Rainer
> > > > > > _______________________________________________
> > > > > > Syslog-sec mailing list
> > > > > > Syslog-sec@www.employees.org
> > > > > > http://www.employees.org/mailman/listinfo/syslog-sec
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > _______________________________________________
> > > > Syslog-sec mailing list
> > > > Syslog-sec@www.employees.org
> > > > http://www.employees.org/mailman/listinfo/syslog-sec
> > > >
> > > > _______________________________________________
> > > > Syslog-sec mailing list
> > > > Syslog-sec@www.employees.org
> > > > http://www.employees.org/mailman/listinfo/syslog-sec
> > > >
> > >
> > >
> >
> >
>
>
> _______________________________________________
> Syslog-sec mailing list
> Syslog-sec@www.employees.org
> http://www.employees.org/mailman/listinfo/syslog-sec
>
_______________________________________________
Syslog-sec mailing list
Syslog-sec@www.employees.org
http://www.employees.org/mailman/listinfo/syslog-sec