OMC
Not logged in



 OMC Data Server - System Overview


  1. Introduction
  2. OMC Data Server functionality
  3. Current status of the OMC Data Server
  4. OMC science analysis processing at CAB
  5. Column description of output FITS files (OMC light curves)
  6. PROBLEMS column description
  7. Known limitations
  8. Other foreseen functionality

Introduction

The Optical Monitoring Camera (OMC) observes the optical emission from the prime targets of the gamma-ray instruments on-board the ESA mission INTEGRAL (launched on Oct. 17, 2002). OMC has also the capability to monitor serendipitously a large number of optically variable sources within its field of view.

Since only a number of OMC sub-windows (typically 100, and in any case less than 228) of 11x11 pixels can be downloaded to Earth, the targets to be monitored by OMC have to be pre-selected on ground. For this purpose, an OMC Input Catalogue were compiled containing:

The OMC Data Server includes all observations from revolutions 11-1660,1662-1669 (publicly available on Jun 02, 2017) and public observations up to revolution 1811. Data prior to revolution 951 are from revision 2 of the INTEGRAL Archive. From revolution 951 onwards revision 3 is being used. The Off-line Science Analysis software OSA 7.0 was used to process the OMC data until revolution 890. In revolution 891 the Off-line Science Analysis software was switched to OSA 9.0, which includes an improved PSF calculation with respect to OSA 7.0, and in revolution 1281 was switched to OSA 10.1 improving the centroiding capabilities in open loop slews and in some extreme cases (crowded fields, faint sources...). Work to update old data to revision 3 and their reprocessing with OSA 10.1 is ongoing.


The OMC Data Server includes completely the OMC Input Catalogue V0005, so the user can check if a given source has been pre-selected as a candidate to be monitored by the OMC. However the main aim of the OMC Data Server is to give a friendly interface to the OMC science results.


OMC Data Server functionality

Archive Search

The OMC catalogue comprises 541802 objects. The query to access the archive is made by means of an HTML fill-in form which permits to perform queries by object name, coordinates, object type, V-magnitude range, date of observation, time binning, centroid method and/or number of points of the light curve. The output data may be ordered by object name, coordinates, magnitude or date and time of observation. Two output formats are available: HTML or ASCII.


The system has a built-in name resolver utility which makes possible to query the archive using any of the object names provided by SIMBAD. The name resolver gives more than three million and a half identifications for the astronomical objects contained in the OMC catalogue. The full list of the names associated to a given object can be obtained by simply clicking on the target name in the output form.


Results from Search

The following utilities are provided in HTML output format to the users with proper access rights:


Current status of the OMC Data Server

The access to the OMC Input Catalogue data is open for everyone. It allows to know which objects have been monitored by the OMC, and when. It also provides a cross-correlation window based on the SIMBAD Database.


Security and privacy of the private data are assured in two ways: user authentication (username+password) and encrypted data transfer. There are several types of user profiles, each of them with different data access policies. The restricted interface to access PV phase and Core Programme data is open only for the INTEGRAL teams. To obtain your password please send an e-mail to omc-support@cab.inta-csic.es.


Currently, you can retrieve from the OMC Data Server the processed light curves. To get the original CCD windows for a manual reprocessing you have to use the standard interface at ISDC. We plan to give also this functionality in the future.


OMC Data Server statistics

Total number of observed sources (at least one photometric point):    236388
Number of "scientific" observed sources (at least one photometric point):    154768
Number of scientific sources with more than P photometric points (time binning of 10 minutes):
Num. points (P)Num. of sources
> 50      87292     
> 100      63236     
> 200      40584     
> 500      17484     
> 1000      6782     
> 2000      1955     
> 3000      744     
> 4000      220     
> 5000      124     

Those sources with more than 50 photometric points (time binning of 10 minutes) and with object type available in the Simbad Database, show the following classification:

Object type Number of sources
Variable Star6697     
Variable Star of Mira Cet type2793     
Variable Star of RR Lyr type2555     
Eclipsing binary of Algol type1840     
Semi-regular pulsating Star1363     
Quasar1165     
Pulsating variable Star1143     
Pulsars652     
Possible Quasar549     
Eclipsing binary499     
Radio Galaxy399     
T Tau-type Star383     
Emission-line galaxy382     
Classical Cepheid (delta Cep type)371     
Eclipsing binary of beta Lyr type358     
X-ray source302     
Eclipsing binary of W UMa type262     
Nova260     
gamma-ray source251     
Flare Star229     
Emission-line Star224     
Classical Cepheid variable Star224     
Variable Star of Orion Type220     
Galaxy220     
Dwarf Nova192     
Low Mass X-ray Binary170     
Seyfert 2 Galaxy165     
Seyfert 1 Galaxy162     
Carbon Star159     
Variable Star of irregular type143     
High Mass X-ray Binary117     
Infra-Red source100     
Emission Object92     
Variable Star with rapid variations88     
BL Lac - type object86     
Active Galaxy Nucleus80     
Star in Cluster79     
Variable Star of RV Tau type53     
Star50     
Seyfert Galaxy45     
Variable Star of W Vir type44     
Variable Star of alpha2 CVn type41     
Be Star41     
Radio-source40     
Variable Star of beta Cep type40     
Nova-like Star39     
Star suspected of Variability35     
Spectroscopic binary34     
Variable Star of delta Sct type34     
HII Galaxy31     
Cataclysmic Variable Star31     
SuperNova Remnant30     
Symbiotic Star28     
Planetary Nebula27     
Variable of RS CVn type26     
Eruptive variable Star18     
Wolf-Rayet Star18     
LINER-type Active Galaxy Nucleus17     
Cataclysmic Var. AM Her type17     
Variable of BY Dra type16     
Cluster of Galaxies16     
X-ray Binary14     
Blue object13     
S Star13     
Variable Star of R CrB type12     
White Dwarf10     
Galaxy in Pair of Galaxies9     
Variable White Dwarf of ZZ Cet type9     
HII (ionized) region8     
UV-emission source8     
Elliptical variable Star7     
Galaxy in Cluster of Galaxies7     
Pair of Galaxies7     
Cataclysmic Var. DQ Her type7     
Rotationally variable Star7     
Star in double system7     
Possible Planetary Nebula6     
Double or multiple star6     
Low Surface Brightness Galaxy5     
High proper-motion Star5     
gamma-ray Burster5     
Gravitationnaly Lensed Image of a Quasar5     
Variable Star of FU Ori type4     
Object of unknown nature4     
SuperNova Remnant Candidate4     
Nebula of unknown nature3     
SuperNova3     
Globular Cluster2     
Blazar2     
Cluster of Stars2     
Star in Association2     
Starburst Galaxy2     
Group of Galaxies2     
Maser2     
Star in Nebula2     
Interacting Galaxies1     
Young Stellar Object1     
Molecular Cloud1     

OMC science analysis processing at CAB

Around 100 OMC CCD sub-windows, each of 11x11 pixels, are downloaded to Earth about every minute, without any on-board processing. The data are received at the Integral Science Data Center, where they are completely processed in an automatic way. OMC data are then re-processed at CAB and stored in the OMC Data Server. The principal steps of this science analysis processing are the following:

  1. The information in the telemetry packages is converted to CCD pixel values.

  2. The OMC CCD sub-windows are corrected from bias and dark current. They are then convolved with the corresponding flat-field matrix.

  3. OMC CCD sub-windows obtained within periods of around 10 minutes (default value) as well as within the full Science Window are combined to obtain a better signal-to-noise ratio. A Science Window is a pointing (typically around 30 minutes) or a part of it in case of being too long (staring mode). A third solution without any combination of the OMC CCD sub-windows is also given. These three cases correspond to three different processing options.

  4. The photometric value for each source of interest is obtained by applying three different extraction masks of 1x1, 3x3 (default value) and 5x5 pixels to the combined images (see column description below). The local background, as measured on a corona around the extraction mask, is subtracted. If Source coordinates (default) is selected as the centroid method in the fill-in form, the position of the extraction mask is re-centred on the source coordinates given in the OMC Input Catalogue, allowing a maximum offset of 10 arcsec (parameter IMA_maxWcsOff in OMC OSA). Instead if Brightest pixel is selected as the centroid method, the position of the extraction mask is re-centered on the brightest pixel within the central 5x5 pixel section. This extraction software can also be run offline to modify the parameters, in order to minimize the effects of background stars when needed.

  5. The photometric values are stored as V magnitudes, thus compiling light curves with three time binnings:

    • Shot-by-Shot: one photometric point per shot.
    • 10 minutes: around one photometric point each 10 minutes. Shots with exposure smaller than 20 seconds are rejected to increase the signal-to-noise ratio.
    • ScW-by-ScW: one photometric point per Science Window. Shots with exposure smaller than 60 seconds are rejected to increase the signal-to-noise ratio.

    We must notice that due to the rejection of the shortest shots when using time binnings of 10 minutes and ScW-by-ScW, the brightest sources will not be processed when selecting these time binnings, because they will be saturated in the remaining shots. For these sources, the user should select the time binning of Shot-by-Shot, and then filter the results to keep only the shortest shots with no saturation effects.

In this science analysis, OMC Input Catalogue Version V0005 and the following versions of OMC OSA 7.0, OSA 9.0 and OSA 10.1 (Off-line Science Analysis) components were used:

OSA 7.0 (revolutions 11-890):

omc_science_analysis 5.3.1
  omc_scw_analysis 5.3.1
    o_cor_science 5.3.1
      o_cor_box_fluxes 6.5.1
    o_gti 5.3.1
      gti_create 2.2
      gti_attitude 1.5
      gti_merge 1.6
    o_src_analysis 5.3.1
      o_src_get_fluxes 8.1
      o_src_compute_mag 4.4
  omc_obs_analysis 5.3.1
    o_src_collect 2.4

OSA 9.0 (revolutions 891-1280):

omc_science_analysis 6.0
  omc_scw_analysis 6.0
    o_cor_science 6.0
      o_cor_box_fluxes 6.5.1
    o_gti 6.0
      gti_create 2.3
      gti_attitude 1.5
      gti_merge 1.6
    o_src_analysis 6.0
      o_src_get_fluxes 8.2
      o_src_compute_mag 4.4
  omc_obs_analysis 6.0
    o_src_collect 2.4

OSA 10.1 (revolutions 1281-->):

omc_science_analysis 6.0.2
  omc_scw_analysis 6.0.2
    o_cor_science 6.0.2
      o_cor_box_fluxes 6.5.1
    o_gti 6.0.2
      gti_create 2.3
      gti_attitude 1.5
      gti_merge 1.6
    o_src_analysis 6.0.2
      o_src_get_fluxes 8.3
      o_src_compute_mag 4.4
  omc_obs_analysis 6.0.2
    o_src_collect 2.4

Best photometric accuracy achievable

Expected best photometric accuracy in V magnitudes of the OMC for different V values and various effective integration times. A typical observation totals 300 s of effective integration time.

V magnitude
Eff. integration (s) 8 10 12 14 16
10       0.007 0.02 0.1 -- --
300       -- 0.005 0.01 0.04 0.3
900       -- 0.003 0.006 0.026 0.17

The above values do not account for possible systematic biases.


Calibration status

In August 2006 a new set of flatfield matrices (Version 5) as well as photometric calibration files (Version 5) were delivered by the OMC team to ISDC. These new IC (Instrument Characteristic) files improved substantially the old ones (Version 4).


The data currently available at the OMC Data Server were analyzed using the new IC files. In the following table the version number of each IC file used in the analysis is given:

IC Data Structure Version
OMC.-PHOT-CAL 5
OMC.-FLAT-CAL 5
OMC.-BDPX-CAL 2
OMC.-DARK-CAL 2
OMC.-GOOD-LIM 7
INTL-GOOD-LIM 2

Column description of output fits files (OMC light curves)

REVOL Revolution number valid for time of data taking
SWID Science Window identifier from which this row was taken
TFIRST Time of the first data element in ISDC Julian days
BARYTIME Barycentric time for the first data element in ISDC Julian days
TELAPSE Elapsed time of the integration in seconds
EXPOSURE Effective integration time in seconds
SHOTTYPE Type of shots used for building integration (1=Photometric, 2=Science)
OMC_ID OMC catalogue source identifier (IOMC number)
TYPE_TAR Target type (1=Photometric, 2=Science, 3=Dark Current)
RA_OBJ Source right ascension in degrees
DEC_OBJ Source declination in degrees
FLUX_1 Flux in electron/s derived from 1x1 integration boxes
ERFLUX_1 Error estimate for FLUX_1
FLUX_3 Flux in electron/s derived from 3x3 integration boxes
ERFLUX_3 Error estimate for FLUX_3
FLUX_5 Flux in electron/s derived from 5x5 integration boxes
ERFLUX_5 Error estimate for FLUX_5
SKYBACK Mean flux from sky background in electron/pixel/s
SKYERROR Error estimate for SKYBACK
SIZE_MAG Integration box size for deriving MAG_V
MAG_V Computed V (Johnson) magnitude (default value)
ERRMAG_V Error estimate for V magnitude
CATMAG_V Catalog V (Johnson) magnitude
CATERR_V Catalog error estimate for V magnitude
MAG_V1 Computed V magnitude for the 1x1 pixel area
ERMAG_V1 Error estimate for V magnitude in 1x1 pixel area
MAG_V3 Computed V magnitude for the 3x3 pixel area
ERMAG_V3 Error estimate for V magnitude in 3x3 pixel area
MAG_V5 Computed V magnitude for the 5x5 pixel area
ERMAG_V5 Error estimate for V magnitude in 5x5 pixel area
PROBLEMS Flag for various problems (0 = no problems)
NOISE_LL Read-out noise in e- (low gain, left ROP)
NOISE_LR Read-out noise in e- (low gain, right ROP)
NOISE_HL Read-out noise in e- (high gain, left ROP)
NOISE_HR Read-out noise in e- (high gain, right ROP)
CENTRING_X Derived X-axis offset of the source from the box centre (pixels)
CENTRING_Y Derived Y-axis offset of the source from the box centre (pixels)
PSF_FWHM Effective PSF FWHM in pixels
X_TAR X coordinate of the lower left pixel of the target box (pixels)
Y_TAR Y coordinate of the lower left pixel of the target box (pixels)
RANK On-board sequence number of box
RA_FIN Derived right ascension in degrees
RA_FIN_ERR Standard error for RA_FIN*cos(DEC_FIN)
DEC_FIN Derived declination in degrees
DEC_FIN_ERR Standard error for DEC_FIN

PROBLEMS column description

This is the meaning of PROBLEMS column in output fits files. Problems are stored in an unsigned integer register, and any problem encountered is logically ANDed to the existing register value. Deconstruction of the total into its only possible component values reveal the individual PROBLEMS. Problem values 64 and 2048 are reserved for future improvements.

NAME VALUE   DESCRIPTION
OMC_PROBLEM_NONE 0   No problems
OMC_PROBLEM_CENTROID_OUT 1   Centroid does not match maxWcsOff radius; The WCS position has been used as centroid
OMC_PROBLEM_EXTRAPOLATED_MAG 2   The magnitude was extrapolated
OMC_PROBLEM_BAD_CENTROID 4   No centroid is available or is inaccurate
OMC_PROBLEM_BAD_PSF 8   Bad PSF. A default value was used
OMC_PROBLEM_ANOMALOUS_PSF 16   The PSF shape is anomalous
OMC_PROBLEM_LOW_FLUX_1 32   Flux of central pixel too low
OMC_PROBLEM_BADPIXEL_SKY 128   Bad pixel found in sky background
OMC_PROBLEM_BADPIXEL_RIM_5 256   Bad pixel found in 5x5 rim
OMC_PROBLEM_BADPIXEL_RIM_3 512   Bad pixel found in 3x3 rim
OMC_PROBLEM_BADPIXEL_RIM_1 1024   Central pixel bad
OMC_PROBLEM_SKY_ERROR 4096   Sky error larger than accepted limit
OMC_PROBLEM_UNKNOWN_MAG 8192   Magnitude could not be calculated
OMC_PROBLEM_EXTND_SRC 16384   Source is extended - flux not valid
OMC_PROBLEM_COORD_OUT 32768   WCS position close to the edge or out of the OMC box; The brightest pixel has been used.

Known limitations


Other foreseen functionalities

A variety of data analysis tools are being developed and will be available in the near future. The aim of these tools is to provide added-value functionalities to the system giving the ability to perform data analysis tasks remotely.


Retrieval of OMC images: The user will be able to get raw and corrected images. In the last case the OMC sub-windows will be corrected for bias, dark current and flatfield.


Time series analysis: The operation of INTEGRAL from a high orbit allows a continuous observation (only interrupted by the radiation belts crossing) for periods of several weeks giving a unique photometric capability that cannot be addressed from ground-based observatories. Our aim is that the OMC data server also provides information in the frequency domain. Given the diversity of the OMC targets (AGNs, X-rays binaries, gamma-ray burst, eclipsing binaries, pulsating variables, ...) and the variability patterns (long/short-term variations, mono/multi-periodicity,...), a detailed study on the techniques to be applied in each case must be performed.


Data mining. Light curve characterization: Even though the OMC data server allows making queries by object type based on the classification provided by SIMBAD, it is clear that this classification can be greatly improved with the use of the OMC data. Given the vast amount of data to be handled, classification procedures based on the visual inspection by experts are not adequate and data mining techniques must be used instead. We are presently working in a neural network system to classify light curves of periodic variable stars. The network will be trained with the HIPPARCOS light curves and, in a first step, it will allow for the automatic classification of eclipsing binary stars with a further extension to other types of variable stars (e.g. pulsating variables). The system is designed to perform a unsupervised topological mapping based on morphological proximity among the light curves.




Version 3.0 - March 2012     © CAB (INTA-CSIC) Home