As, unfortunately, it is not possible to accommodate discussion about
Visualization in the ATF agenda for coming meetings, Stephen has asked me
to write a summary of the current thinking and status of the work in the
Atlas Graphics Group. In the following text, I'm not going into details
as most topics are already documented on the Web. Also, it's not completely
clear which topics are ATF members interested in most (high level philosophicaly-architectural
questions, current status, plans, description of components, technicalities,...).
So, if you have any question, just ask it.
Visualization for Atlas
-
What is Visualization ?
-
What kinds of Visualization we see today ?
-
What is the Glue ?
-
What are the relations to other Domains ?
-
What is the current situation and plans ?
-
What is the comparison to similar HEP projects ?
-
What are the problems ?
note:
The most relevant references are annotated
as (*).
What is Visualization ?
Let us suppose that we will build our software using OO approach. This
means, that our software will consist of Objects. Pure Objects (in whatever
programing language) are not very user-friendly, they can only be accessed
in the programming way. Sometimes, this access can be quite tricky and
the core object interface may be very technical for the real (physicists)
access. So we need some way how to make access to objects easier and more
useful. We need some way how to inspect objects and interact with them.
As we are in the OO world, this inspection and interaction will be done
again via (other) objects. And as observers should not (unintentionally)
influence the running of the program, inspection and interaction objects
should not be part of the core software, they should be additional. They
will represent the original object to the user. User wants various ways
of inspection and interaction with objects, he needs various Representations.
Those various representations of course share something, they share the
original object's Model. From the point of view of the relation
to the original object, there are several types of Representations:
one-to-one (Event Display), one-to-many (Detector Display), many-to-one
(Histograms). Representations themselves are again objects, they
may have their own Representations (Chain of Representation).
When several objects are represented in the same way, they share the
Scene.
The Scene is the interface between the program and the user. It
is possible, that several kinds of Representations share the same
Scene.
Scenes
can be grouped according to the nature of the Representation
they
interface. Representations of the same object are interchangeable
within the group and they can be (in principle) generated one from another
(Democracy of Scenes). Scenes are very often implemented
by the external packages, which will change and disappear, new packages
(and Scenes) will appear. Requirements to visualization are also
changing due mutual interaction between the users and the package abilities
and improvement in the physics understanding. This means that the whole
system should not depend on Scenes, it should be possible to change
Scenes,
remove some and add others.
Translated into more technical terms, it can be described like this
(simplified): For every object which has to be visualized, there should
be an aditional object (Representations) for each kind of visualization.
This Representation
should be external to the original object. Representations
of the same kind are visualized using another object (or set of objects)
-
Scene. Scene implements the particular visualization protocol,
very often it's just the interface to some external (to Atlas) application.
The whole Graphics Architecture is derived from this Analysis. The Graphics
Design and Implementation is evolving so it has at any moment both running
version and plans for further development.
note:
See Aravis
documentation (*) for more user-friendly
description of the current design based on described ideas.
See Atlas
Graphics Design (and discussions during its Review) for detailed description
of the earlier version of that design.
See Conceptual
Design class diagram (*) for example of
the real classes.
See Atlas
Graphics Tutorial for slightly outdated description of the implementation
of the design.
note:
Dictionary (will be consolidated):
| Here |
Design, Code |
Requirements |
| object |
Plottable |
Real Object |
| Model |
(Plottable)Model |
|
| Representation |
(Plottable)Rep |
Graphical Object |
| Scene |
(Abstract)Scene |
View |
What kinds of Visualization we see today ?
This list is by far not exhaustive:
-
Event Display (plus Detector Display) - This is traditionally the
main kind of the Visualization.
-
Simple - It is used for the software tuning and debugging. It should
be simple and easy to use for the software developers.
-
3D - It is traditionally the main kind of the Event Display. Its
usefulness for the real work is quite limited, except for getting the overall
feeling of the Event. It is useful for publicity. It should be very easy
to use.
-
Physics - It is useful for real physical study. It offers special
mappings, projections and manipulations exposing the real physics properties
of the Event. The use of it requires the deep understanding of the physics
ideas behind.
-
Statistics (many-to-one Representation) - This is the visualization
of the statistics (analysis) objects.
-
Histograms - They are the real end of the statistics analysis process.
There are only limited ways of manipulation of histograms.
-
N-tuples and Trees - They are intermediate objects of the analysis
process. They are still subject to further manipulations, the results of
which are represented as histograms (Chain of Representation: object
-> n-tuple -> histogram -> visual histogram,...).
-
User Interface - In the OO program, also commands are objects and
user interacts with their representations.
-
Command-line - It's the simple interface to commands (and subsequently
scripting).
-
GUI - It's higher level interface to commands.
-
Query Interface - It's the interface to the commands of the analysis
process. Can be Command-line or GUI.
-
Misc - There are other kinds of the object representations.
-
Logging - Log-file contains just the textual representation of the
objects.
-
Data Exchange - Very often, it's useful to export/import data in
some convenient data file. It again contains just the object representations.
What is the Glue ?
Glue is the software that puts all visualization components together and
allows their collaboration. It is the control subdomain of the Graphics
domain. Even the Glue should be evolvable and we should not regard any
of its part as definitive. Glue makes sure that all concerned objects (Representations,
Scenes,...)
"speak the same language" - it defines the standard API (Application Programing
Interface) for them. This standard interface further allows to use various
ways of accessing the visualization (i.e. the creation of real applications).
What are the relations to other Domains ?
-
Event, SubSystems - They are the primary providers of the data for
Visualization. At the same time, they make requirements and collaborate
in the definition of the particular ways of displaying those data.
-
Reconstruction - It uses Visualization as the interface to the user.
The collaboration with the Visualization Domain is similar to that of Event
and SubSystems Domains.
-
Control - It provides the environment for the Visualization to work.
-
Analysis - It operates on data via queries, agents, information
extraction, data mining,... Visualization enables analysis to perform such
operations.
What is the current situation and plans ?
Let's shortly mention only most important Scenes:
-
Event Display
-
Simple
-
Aravis
- interactive, 80% done
The original Arve graphics (T.Burnett) has been taken out of Arve and
partly re-written (R.&D.Candlin) so it is now the standard Scene
and can be used via the standard interface. It doesn't do any fancy graphics,
but it's simple to use (even from the C++ point of view). It is useful
to debug geometry (in fact, it has already helped to find several bugs
in the Detector Description).
-
3D
-
VRML - standalone, 100% done
It is possible to write out the VRML 1.0 files which describe 3D view
of the data. Those files can be then read into several browsers. It's useful
for simple pictures.
-
OpenInventor / HEPVis
/ OpenScientist -
interactive, standalone, 0% done
Open Inventor is the C++ library of 3D objects. HEPVis is the C++ library
of HEP 3D objects build on top of Open Inventor. It's a natural candidate
for an interactive 3D Event Display.
-
Physics
-
Atlantis - standalone, 80% done
It is highly sophisticated Event Display for real physics use (H.Drevermann
& comp.). It has been developed from the Aleph Event Display - Dali.
It offers special projections and manipulations useful to really see the
physics inside Event. It's written in Fortran (plus a bit of C and C++).
The interface to the rest is done via XML files. It's not easy to use,
the user should understand what she's doing.
-
Wired - standalone, 90% done
It is potentially a similar tool as Atlantis. Today, it doesn't have
yet all the functionality of Atlantis. It is written in Java. The interface
to the rest is via XML files. It's the common HEP project, the core developer
(M.Donszelmann) is in CERN/IT/IPT and there are many collaborators from
other experiments and labs (D.Koper for Atlas). It is an external
application (*) to
Atlas.
-
Statistics
-
Histograms
-
LHC++ / HTL
- standalone, 80% done
HTL offers 1d or 2d histograms (no n-tuples). It uses heavily STL.
It's supposed to run with the Objectivity in the background (and then it
can feed histograms into HEP Explorer) - this feature is not (yet) supported
in Atlas.
-
OpenScientist - interactive, standalone, 0% done
Open Scientist (G.Barrand) offers very nice histograms and n-tuples,
which could be exported into Objectivity or Root (using Rio interface,
which is independent of Root itself). They can be visualized also by the
OpenInventor browser. It's another kind of the common HEP project. Open
Scientist offers also other functionality (other graphics, scripting,...).
-
N-tuples
-
HBook - standalone, 100% done
It is the interface to the known Zebra hbook files. It can just write
out ntuples, which can be then read by Paw or JAS.
-
OpenScientist - interactive, standalone, 0% done
see above
-
User Interface - 0% done
-
Misc
-
Logging
-
AsciiText - 100% done
It just allows logging of objects. For each object, its textual description
is written to a file.
-
Data Exchange
-
XML
- 90% done
We recommend(*)
to use XML as the text-file format for data (and document)
exchange. XML is now the main bridge between core software and Atlantis
and Wired. The proper XML file format (dtd) is currently under discussion.
Currently, we see those candidates for access to visualization via standard
Glue (they are not completely orthogonal):
-
Scene List - 100% done
It is the simple, programming - only interface for handling Scenes.
-
Object Browser - 40% done
It uses file=object, directory=container analogy and allows easy invocation
of Scenes for selection of objects. The mechanism and prototype
exist (L.Tuura).
-
Components - 0% done
It is very natural to see a Scene as a Component (in
the
Control Component Network). It just accepts any object and makes
proper Representation of it.
Concerning applications, Graphics could be used from any C++ program. The
only requirement is having objects for the display available. Currently
there are two main applications:
-
GraphicsFromEvent
- It is the test program from Event/EventManagement equipped with the visualization.
There is also the prototype of the WWW interface to it.
-
T2Ref Analysis - It is the standard T2Ref b-physics reconstruction
program equipped with the visualization.
The data sources have to be available to the Graphics Domain by other Domains
and SubSystems (in principle):
-
Offline
-
InnerDetector
-
exist: SiDetector, TRT_Detector, SiDigit, TRT_Digit, StripCluster
-
visualization: all existing objects have full visualization interface
-
contact in SubSystem: identified (M.Tadel)
-
MuonDetector
-
exist: MDT_Digit, MDT_Detector
-
visualization: no visualization done yet
-
contact in SubSystem: 0
-
LArg
-
exist: CaloDigit
-
visualization: no visualization done yet
-
contact in SubSystem: 0
-
TileCal
-
exist: 0
-
visualization: NA
-
contact in SubSystem: identified (S.Nemecek)
-
Simulation
-
exist: TruthTrack, TruthVertex
-
visualization: done for TruthTrack
-
contact in SubSystem: 0
-
Reconstruction:
-
exist: 0
-
visualization: NA
-
contact in SubSystem: 0
-
T2Ref
-
exist: TrtHit, TrtTrack, PrecisionHit, PrecisionTrack,...
-
visualization: mentioned objects have full visualization interface
-
contact in SubSystem: identified (M.Sessler)
There are some supporting applications:
-
CreatePlottableModel
- Wizard-like script for creation of all graphics classes necessary
to add the visualization to any existing class.
-
ExpatInterface, XMLTools - C++ and Fortran interface to the
XML parser Expat.
-
DependencyGrapher
- Application showing tree of dependencies between SRT packages. It's also
available from the Web
.
All those component are documented on the Web(*).
Also, there is detailed documentation of everything in the Graphics Domain,
which covers all phases of the development cycle: Requirements (All
Requirements, including technical ones and some belonging to other Domains(*),
Explained
Requirements for Event Display (*)),
Design, Code, Documentation (Implementation
Guidelines , Aravis
documentation) and Gallery.
Most of the documentation is automatically created from the sources in
CVS.
There are other graphical applications in the Atlas traditional (legacy)
software (MuonBox, xKalman,...). They are currently not included in this
described schema. The reuse of expertise from those applications is highly
desirable as they often contain deep physics and detector knowledge and
useful concepts (for example the GUI in the MuonBox). The interface
via XML files could be easy, direct interface (via wrapping) could be possible
if authors are willing to cooperate and applications are reasonably modular
(modularity has two conditions: to be able to accommodate other modules
and to be able to act as a module in something else).
note:
See Top
level Domain Decomposition (*) of the
Graphics Domain.
See Existing
Scenes (*) overview.
See Data
Sources in the offline software (*).
See Data
Sources in the t2ref software (*).
What is the comparison to similar HEP projects ?
Gaudi
The schema is very similar. Generally, the data objects + converters
concept serves the same purpose as our schema. Gaudi Event Display (chapter
3.8) talks about data objects, their graphical representation
(= Representations) and Converters (= Scenes). Also
Gaudi Histogramming (chapter 14,15) talks about histogram data representation.
Here, our approach is more OO, as it makes statistical entities just representations
of objects. Concerning UI (chapter 16), Gaudi touches the same similarity
by defining the event model for it, but neither project is very
elaborated at this moment.
LHC++
LHC++ doesn't define specific Event Display architecture. Its Histogramming
architecture (HTL) is coupled to the data storage (Objectivity). It also
doesn't offer n-tuples, replacing them with tag databases (which are then
very tightly coupled to the whole storage schema). We can reuse LHC++ components
(like HTL) in our system.
Root
Root goes even further than LHC++ in building tightly coupled architecture,
where everything depends on everything else. In particular, all objects
themselves have access to the storage (RootIO) and rendering (Root), which
makes using of other storage or visual packages complicated, while it brings
certain advantage in the easiness of using those default facilities. We
can use Root as independent application by exporting files via Open Scientist
Rio.
Open Scientist
Open Scientist
concept (*) is very
similar to our histograms and n-tuples. Themselves, they are independent
on any storage or visual mechanism. Access to the storage and rendering
is external to them, Adapter Pattern is used. In practice, it means
that everything could be stored in both Objectivity and Root (via Root
independent packages - Rio) and displayed using various techniques (Open
Inventor is preferred). The Open Scientist approach is very compatible
with ours - they can live together.
What are the problems ?
Most problems and not specific to this Domain, they are common to the whole
Atlas offline software:
-
There is a lack of interaction with SubSystems. Visualization has not been
mentioned in the Review, it doesn't exist in the Management plan, it seems
to be up to now ignored by the ATF. There are collaborating individuals
in SubSystems, but no official contacts are foreseen by the Management.
-
Visualization is on the end of the chain of all other Domains, it can only
display objects which exist. We still have only limited number of useful
objects from other Domains (this is connected to the previous point).
-
There is no public forum for discussing interfaces to other Domains. DIG
has been killed, no replacement has been created.
-
There are many technical software items, like use of ExceptionHandling,
Pre/PostConditions, Environment Setup, Graphical Libraries,... but no forum
to discuss and decide on them.
-
Java is quite probable the future environment at least for doing Visualization
and Analysis. We should start looking at it (and possibly define the Java
Pilot Project).
-
There is the lack of manpower for mission-critical, but boring tasks (like
infrastructure). The only reward one gets for working in the offline software
is the fun. This means that no-one is interested in developing boring pieces
and in reusing and maintaining code written by others.
For further documentation of the Atlas Graphics, look at http://atlasinfo.cern.ch/Atlas/GROUPS/GRAPHICS/Texts/EventDisplay/.
The Atlas Graphics Group home is at http://atlasinfo.cern.ch/Atlas/GROUPS/GRAPHICS/.
J.Hrivnac, Atlas
Graphics Group for Atlas
Architectural Tasks Force, July'99