As, unfortunately, it is not possible to accommodate discussion about Visualization in the ATF agenda for coming meetings, Stephen has asked me to write a summary of the current thinking and status of the work in the Atlas Graphics Group. In the following text, I'm not going into details as most topics are already documented on the Web. Also, it's not completely clear which topics are ATF members interested in most (high level philosophicaly-architectural questions, current status, plans, description of components, technicalities,...). So, if you have any question, just ask it.


Visualization for Atlas

  1. What is Visualization ?
  2. What kinds of Visualization we see today ?
  3. What is the Glue ?
  4. What are the relations to other Domains ?
  5. What is the current situation and plans ?
  6. What is the comparison to similar HEP projects ?
  7. What are the problems ?
note:
 The most relevant references are annotated as (*).

What is Visualization ?

Let us suppose that we will build our software using OO approach. This means, that our software will consist of Objects. Pure Objects (in whatever programing language) are not very user-friendly, they can only be accessed in the programming way. Sometimes, this access can be quite tricky and the core object interface may be very technical for the real (physicists) access. So we need some way how to make access to objects easier and more useful. We need some way how to inspect objects and interact with them.

As we are in the OO world, this inspection and interaction will be done again via (other) objects. And as observers should not (unintentionally) influence the running of the program, inspection and interaction objects should not be part of the core software, they should be additional. They will represent the original object to the user. User wants various ways of inspection and interaction with objects, he needs various Representations. Those various representations of course share something, they share the original object's Model. From the point of view of the relation to the original object, there are several types of Representations: one-to-one (Event Display), one-to-many (Detector Display), many-to-one (Histograms). Representations themselves are again objects, they may have their own Representations (Chain of Representation).

When several objects are represented in the same way, they share the Scene. The Scene is the interface between the program and the user. It is possible, that several kinds of Representations share the same Scene. Scenes can be grouped according to the nature of the Representation they interface. Representations of the same object are interchangeable within the group and they can be (in principle) generated one from another (Democracy of Scenes). Scenes are very often implemented by the external packages, which will change and disappear, new packages (and Scenes) will appear. Requirements to visualization are also changing due mutual interaction between the users and the package abilities and improvement in the physics understanding. This means that the whole system should not depend on Scenes, it should be possible to change Scenes, remove some and add others.

Translated into more technical terms, it can be described like this (simplified): For every object which has to be visualized, there should be an aditional object (Representations) for each kind of visualization. This Representation should be external to the original object. Representations of the same kind are visualized using another object (or set of objects) - Scene. Scene implements the particular visualization protocol, very often it's just the interface to some external (to Atlas) application.

The whole Graphics Architecture is derived from this Analysis. The Graphics Design and Implementation is evolving so it has at any moment both running version and plans for further development.

note:
See Aravis documentation (*) for more user-friendly description of the current design based on described ideas.
See Atlas Graphics Design (and discussions during its Review) for detailed description of the earlier version of that design.
See Conceptual Design class diagram (*) for example of the real classes.
See Atlas Graphics Tutorial for slightly outdated description of the implementation of the design.

note:
Dictionary (will be consolidated):
Here Design, Code Requirements
object Plottable Real Object
Model (Plottable)Model
Representation (Plottable)Rep Graphical Object
Scene (Abstract)Scene View



 

What kinds of Visualization we see today ?

This list is by far not exhaustive:
  1. Event Display (plus Detector Display) - This is traditionally the main kind of the Visualization.
    1. Simple - It is used for the software tuning and debugging. It should be simple and easy to use for the software developers.
    2. 3D - It is traditionally the main kind of the Event Display. Its usefulness for the real work is quite limited, except for getting the overall feeling of the Event. It is useful for publicity. It should be very easy to use.
    3. Physics - It is useful for real physical study. It offers special mappings, projections and manipulations exposing the real physics properties of the Event. The use of it requires the deep understanding of the physics ideas behind.
  2. Statistics (many-to-one Representation) - This is the visualization of the statistics (analysis) objects.
    1. Histograms - They are the real end of the statistics analysis process. There are only limited ways of manipulation of histograms.
    2. N-tuples and Trees - They are intermediate objects of the analysis process. They are still subject to further manipulations, the results of which are represented as histograms (Chain of Representation: object -> n-tuple -> histogram -> visual histogram,...).
  3. User Interface - In the OO program, also commands are objects and user interacts with their representations.
    1. Command-line - It's the simple interface to commands (and subsequently scripting).
    2. GUI - It's higher level interface to commands.
    3. Query Interface - It's the interface to the commands of the analysis process. Can be Command-line or GUI.
  4. Misc - There are other kinds of the object representations.
    1. Logging - Log-file contains just the textual representation of the objects.
    2. Data Exchange - Very often, it's useful to export/import data in some convenient data file. It again contains just the object representations.


What is the Glue ?

Glue is the software that puts all visualization components together and allows their collaboration. It is the control subdomain of the Graphics domain. Even the Glue should be evolvable and we should not regard any of its part as definitive. Glue makes sure that all concerned objects (Representations, Scenes,...) "speak the same language" - it defines the standard API (Application Programing Interface) for them. This standard interface further allows to use various ways of accessing the visualization (i.e. the creation of real applications).

What are the relations to other Domains ?

  1. Event, SubSystems - They are the primary providers of the data for Visualization. At the same time, they make requirements and collaborate in the definition of the particular ways of displaying those data.
  2. Reconstruction - It uses Visualization as the interface to the user. The collaboration with the Visualization Domain is similar to that of Event and SubSystems Domains.
  3. Control - It provides the environment for the Visualization to work.
  4. Analysis - It operates on data via queries, agents, information extraction, data mining,... Visualization enables analysis to perform such operations.


What is the current situation and plans ?

Let's shortly mention only most important Scenes:
  1. Event Display
    1. Simple
      1. Aravis - interactive, 80% done

      2. The original Arve graphics (T.Burnett) has been taken out of Arve and partly re-written (R.&D.Candlin) so it is now the standard Scene and can be used via the standard interface. It doesn't do any fancy graphics, but it's simple to use (even from the C++ point of view). It is useful to debug geometry (in fact, it has already helped to find several bugs in the Detector Description).
    2. 3D
      1. VRML - standalone, 100% done

      2. It is possible to write out the VRML 1.0 files which describe 3D view of the data. Those files can be then read into several browsers. It's useful for simple pictures.
      3. OpenInventor / HEPVis / OpenScientist - interactive, standalone, 0% done

      4. Open Inventor is the C++ library of 3D objects. HEPVis is the C++ library of HEP 3D objects build on top of Open Inventor. It's a natural candidate for an interactive 3D Event Display.
    3. Physics
      1. Atlantis - standalone, 80% done

      2. It is highly sophisticated Event Display for real physics use (H.Drevermann & comp.). It has been developed from the Aleph Event Display - Dali. It offers special projections and manipulations useful to really see the physics inside Event. It's written in Fortran (plus a bit of C and C++). The interface to the rest is done via XML files. It's not easy to use, the user should understand what she's doing.
      3. Wired - standalone, 90% done

      4. It is potentially a similar tool as Atlantis. Today, it doesn't have yet all the functionality of Atlantis. It is written in Java. The interface to the rest is via XML files. It's the common HEP project, the core developer (M.Donszelmann) is in CERN/IT/IPT and there are many collaborators from other experiments and labs (D.Koper for Atlas). It is an external application (*) to Atlas.
  2. Statistics
    1. Histograms
      1. LHC++ / HTL - standalone, 80% done

      2. HTL offers 1d or 2d histograms (no n-tuples). It uses heavily STL. It's supposed to run with the Objectivity in the background (and then it can feed histograms into HEP Explorer) - this feature is not (yet) supported in Atlas.
      3. OpenScientist - interactive, standalone, 0% done

      4. Open Scientist (G.Barrand) offers very nice histograms and n-tuples, which could be exported into Objectivity or Root (using Rio interface, which is independent of Root itself). They can be visualized also by the OpenInventor browser. It's another kind of the common HEP project. Open Scientist offers also other functionality (other graphics, scripting,...).
    2. N-tuples
      1. HBook - standalone, 100% done

      2. It is the interface to the known Zebra hbook files. It can just write out ntuples, which can be then read by Paw or JAS.
      3. OpenScientist - interactive, standalone, 0% done

      4. see above
  3. User Interface - 0% done
  4. Misc
    1. Logging
      1. AsciiText - 100% done

      2. It just allows logging of objects. For each object, its textual description is written to a file.
    2. Data Exchange
      1. XML - 90% done

      2. We recommend(*) to use XML as the text-file format for data (and document) exchange. XML is now the main bridge between core software and Atlantis and Wired. The proper XML file format (dtd) is currently under discussion.
Currently, we see those candidates for access to visualization via standard Glue (they are not completely orthogonal):
  1. Scene List - 100% done

  2. It is the simple, programming - only interface for handling Scenes.
  3. Object Browser - 40% done

  4. It uses file=object, directory=container analogy and allows easy invocation of Scenes for selection of objects. The mechanism and prototype exist (L.Tuura).
  5. Components - 0% done

  6. It is very natural to see a Scene as a Component (in the Control Component Network). It just accepts any object and makes proper Representation of it.
Concerning applications, Graphics could be used from any C++ program. The only requirement is having objects for the display available. Currently there are two main applications:
  1. GraphicsFromEvent - It is the test program from Event/EventManagement equipped with the visualization. There is also the prototype of the WWW interface to it.
  2. T2Ref Analysis - It is the standard T2Ref b-physics reconstruction program equipped with the visualization.
The data sources have to be available to the Graphics Domain by other Domains and SubSystems (in principle):
  1. Offline
    1. InnerDetector
      1. exist: SiDetector, TRT_Detector, SiDigit, TRT_Digit, StripCluster
      2. visualization: all existing objects have full visualization interface
      3. contact in SubSystem: identified (M.Tadel)
    2. MuonDetector
      1. exist: MDT_Digit, MDT_Detector
      2. visualization: no visualization done yet
      3. contact in SubSystem: 0
    3. LArg
      1. exist: CaloDigit
      2. visualization: no visualization done yet
      3. contact in SubSystem: 0
    4. TileCal
      1. exist: 0
      2. visualization: NA
      3. contact in SubSystem: identified (S.Nemecek)
    5. Simulation
      1. exist: TruthTrack, TruthVertex
      2. visualization: done for TruthTrack
      3. contact in SubSystem: 0
    6. Reconstruction:
      1. exist: 0
      2. visualization: NA
      3. contact in SubSystem: 0
  2. T2Ref
    1. exist: TrtHit, TrtTrack, PrecisionHit, PrecisionTrack,...
    2. visualization: mentioned objects have full visualization interface
    3. contact in SubSystem: identified (M.Sessler)
There are some supporting applications:
  1. CreatePlottableModel - Wizard-like script  for creation of all graphics classes necessary to add the visualization to any existing class.
  2. ExpatInterface, XMLTools - C++ and Fortran interface to the XML parser Expat.
  3. DependencyGrapher - Application showing tree of dependencies between SRT packages. It's also available from the Web .
All those component are documented on the Web(*). Also, there is detailed documentation of everything in the Graphics Domain, which covers all phases of the development cycle: Requirements (All Requirements, including technical ones and some belonging to other Domains(*), Explained Requirements for Event Display (*)), Design, Code, Documentation (Implementation Guidelines , Aravis documentation) and Gallery. Most of the documentation is automatically created from the sources in CVS.

There are other graphical applications in the Atlas traditional (legacy) software (MuonBox, xKalman,...). They are currently not included in this described schema. The reuse of expertise from those applications is highly desirable as they often contain deep physics and detector knowledge and  useful concepts (for example the GUI in the MuonBox).  The interface via XML files could be easy, direct interface (via wrapping) could be possible if authors are willing to cooperate and applications are reasonably modular (modularity has two conditions: to be able to accommodate other modules and to be able to act as a module in something else).

note:
See Top level Domain Decomposition (*) of the Graphics Domain.
See Existing Scenes (*) overview.
See Data Sources in the offline software (*).
See Data Sources in the t2ref software (*).


What is the comparison to similar HEP projects ?


Gaudi
The schema is very similar. Generally, the data objects + converters concept serves the same purpose as our schema. Gaudi Event Display (chapter 3.8) talks about data objects, their graphical representation (= Representations) and Converters (= Scenes). Also Gaudi Histogramming (chapter 14,15) talks about histogram data representation. Here, our approach is more OO, as it makes statistical entities just representations of objects. Concerning UI (chapter 16), Gaudi touches the same similarity by defining the event model for it, but neither project is very elaborated at this moment.

LHC++
LHC++ doesn't define specific Event Display architecture. Its Histogramming architecture (HTL) is coupled to the data storage (Objectivity). It also doesn't offer n-tuples, replacing them with tag databases (which are then very tightly coupled to the whole storage schema). We can reuse LHC++ components (like HTL) in our system.

Root
Root goes even further than LHC++ in building tightly coupled architecture, where everything depends on everything else. In particular, all objects themselves have access to the storage (RootIO) and rendering (Root), which makes using of other storage or visual packages complicated, while it brings certain advantage in the easiness of using those default facilities. We can use Root as independent application by exporting files via Open Scientist Rio.

Open Scientist
Open Scientist concept (*) is very similar to our histograms and n-tuples. Themselves, they are independent on any storage or visual mechanism. Access to the storage and rendering is external to them,  Adapter Pattern is used. In practice, it means that everything could be stored in both Objectivity and Root (via Root independent packages - Rio) and displayed using various techniques (Open Inventor is preferred). The Open Scientist approach is very compatible with ours - they can live together.


What are the problems ?

Most problems and not specific to this Domain, they are common to the whole Atlas offline software:
  1. There is a lack of interaction with SubSystems. Visualization has not been mentioned in the Review, it doesn't exist in the Management plan, it seems to be up to now ignored by the ATF. There are collaborating individuals in SubSystems, but no official contacts are foreseen by the Management.
  2. Visualization is on the end of the chain of all other Domains, it can only display objects which exist. We still have only limited number of useful objects from other Domains (this is connected to the previous point).
  3. There is no public forum for discussing interfaces to other Domains. DIG has been killed, no replacement has been created.
  4. There are many technical software items, like use of ExceptionHandling, Pre/PostConditions, Environment Setup, Graphical Libraries,... but no forum to discuss and decide on them.
  5. Java is quite probable the future environment at least for doing Visualization and Analysis. We should start looking at it (and possibly define the Java Pilot Project).
  6. There is the lack of manpower for mission-critical, but boring tasks (like infrastructure). The only reward one gets for working in the offline software is the fun. This means that no-one is interested in developing boring pieces and in reusing and maintaining code written by others.

For further documentation of the  Atlas Graphics, look at http://atlasinfo.cern.ch/Atlas/GROUPS/GRAPHICS/Texts/EventDisplay/.
The Atlas Graphics Group home is at http://atlasinfo.cern.ch/Atlas/GROUPS/GRAPHICS/.

J.Hrivnac, Atlas Graphics Group  for Atlas Architectural Tasks Force, July'99