OMC
Not logged in



 OMC Data Server - System Overview


  1. Introduction
  2. OMC Data Server functionality
  3. Current status of the OMC Data Server
  4. OMC science analysis processing at CAB
  5. Column description of output FITS files (OMC light curves)
  6. PROBLEMS column description
  7. Known limitations
  8. Other foreseen functionality

Introduction

The Optical Monitoring Camera (OMC) observes the optical emission from the prime targets of the gamma-ray instruments on-board the ESA mission INTEGRAL (launched on Oct. 17, 2002). OMC has also the capability to monitor serendipitously a large number of optically variable sources within its field of view.

Since only a number of OMC sub-windows (typically 100, and in any case less than 228) of 11x11 pixels can be downloaded to Earth, the targets to be monitored by OMC have to be pre-selected on ground. For this purpose, an OMC Input Catalogue were compiled containing:

The OMC Data Server includes all observations from revolutions 11-2508, 2510-2524 (publicly available on Sep 01, 2023) and public observations up to revolution 2674. Data prior to revolution 951 are from revision 2 of the INTEGRAL Archive. From revolution 951 onwards revision 3 is being used. The Off-line Science Analysis software OSA 7.0 was used to process the OMC data until revolution 890. In revolution 891 the Off-line Science Analysis software was switched to OSA 9.0, which includes an improved PSF calculation with respect to OSA 7.0, and in revolution 1281 was switched to OSA 10.1 improving the centroiding capabilities in open loop slews and in some extreme cases (crowded fields, faint sources...). Concerning the OMC science analysis, OSA 10.1 is equivalent to the latest software (OSA 11.0). Work to update old data to revision 3 and their reprocessing with OSA 10.1 is ongoing.


The OMC Data Server includes completely the OMC Input Catalogue V0005, so the user can check if a given source has been pre-selected as a candidate to be monitored by the OMC. However the main aim of the OMC Data Server is to give a friendly interface to the OMC science results.


OMC Data Server functionality

Archive Search

The OMC catalogue comprises 541802 objects. The query to access the archive is made by means of an HTML fill-in form which permits to perform queries by object name, coordinates, object type, V-magnitude range, date of observation, time binning, centroid method and/or number of points of the light curve. The output data may be ordered by object name, coordinates, magnitude or date and time of observation. Two output formats are available: HTML or ASCII.


The system has a built-in name resolver utility which makes possible to query the archive using any of the object names provided by SIMBAD. The name resolver gives more than three million and a half identifications for the astronomical objects contained in the OMC catalogue. The full list of the names associated to a given object can be obtained by simply clicking on the target name in the output form.


Results from Search

The following utilities are provided in HTML output format to the users with proper access rights:


Current status of the OMC Data Server

The access to the OMC Input Catalogue data is open for everyone. It allows to know which objects have been monitored by the OMC, and when. It also provides a cross-correlation window based on the SIMBAD Database.


Security and privacy of the private data are assured in two ways: user authentication (username+password) and encrypted data transfer. There are several types of user profiles, each of them with different data access policies. The restricted interface to access PV phase and Core Programme data is open only for the INTEGRAL teams. To obtain your password please send an e-mail to omc-support@cab.inta-csic.es.


Currently, you can retrieve from the OMC Data Server the processed light curves. To get the original CCD windows for a manual reprocessing you have to use the standard interface at ISDC. We plan to give also this functionality in the future.


OMC Data Server statistics

Total number of observed sources (at least one photometric point):    281128
Number of "scientific" observed sources (at least one photometric point):    185676
Number of scientific sources with more than P photometric points (time binning of 10 minutes):
Num. points (P)Num. of sources
> 50      105191     
> 100      77085     
> 200      51333     
> 500      23805     
> 1000      11069     
> 2000      4140     
> 3000      2084     
> 4000      1130     
> 5000      611     

Those sources with more than 50 photometric points (time binning of 10 minutes) and with object type available in the Simbad Database, show the following classification:

Object type Number of sources
Variable Star7741     
Variable Star of Mira Cet type3310     
Variable Star of RR Lyr type3254     
Eclipsing binary of Algol type2076     
Semi-regular pulsating Star1638     
Quasar1512     
Pulsating variable Star1332     
Possible Quasar849     
Pulsars711     
Eclipsing binary569     
Radio Galaxy530     
Emission-line galaxy438     
T Tau-type Star401     
Eclipsing binary of beta Lyr type394     
Classical Cepheid (delta Cep type)390     
X-ray source317     
Eclipsing binary of W UMa type307     
Nova275     
gamma-ray source274     
Flare Star269     
Galaxy264     
Classical Cepheid variable Star252     
Variable Star of Orion Type245     
Emission-line Star238     
Dwarf Nova221     
Seyfert 1 Galaxy205     
Seyfert 2 Galaxy203     
Low Mass X-ray Binary175     
Carbon Star172     
Variable Star of irregular type161     
Emission Object144     
High Mass X-ray Binary117     
Infra-Red source110     
BL Lac - type object103     
Variable Star with rapid variations101     
Active Galaxy Nucleus100     
Star in Cluster83     
HII Galaxy64     
Variable Star of RV Tau type62     
Star60     
Seyfert Galaxy56     
Variable Star of W Vir type54     
Radio-source51     
Nova-like Star46     
Variable Star of alpha2 CVn type44     
Be Star43     
Variable Star of beta Cep type43     
Variable Star of delta Sct type39     
Star suspected of Variability38     
Spectroscopic binary37     
Cataclysmic Variable Star36     
Symbiotic Star33     
Variable of RS CVn type32     
Planetary Nebula31     
SuperNova Remnant31     
Cataclysmic Var. AM Her type22     
Eruptive variable Star20     
Cluster of Galaxies19     
LINER-type Active Galaxy Nucleus19     
Wolf-Rayet Star19     
Variable Star of R CrB type17     
Blue object17     
Variable of BY Dra type16     
UV-emission source15     
S Star14     
X-ray Binary14     
White Dwarf14     
Galaxy in Pair of Galaxies11     
Variable White Dwarf of ZZ Cet type11     
Cataclysmic Var. DQ Her type11     
HII (ionized) region10     
Star in double system8     
Galaxy in Cluster of Galaxies8     
Elliptical variable Star8     
Pair of Galaxies8     
Possible Planetary Nebula8     
Rotationally variable Star7     
Gravitationnaly Lensed Image of a Quasar6     
Double or multiple star6     
Low Surface Brightness Galaxy5     
gamma-ray Burster5     
High proper-motion Star5     
Variable Star of FU Ori type4     
Object of unknown nature4     
SuperNova Remnant Candidate4     
Nebula of unknown nature3     
Star in Nebula3     
SuperNova3     
Group of Galaxies3     
Cluster of Stars2     
Blazar2     
Star in Association2     
Maser2     
Starburst Galaxy2     
Globular Cluster2     
Young Stellar Object1     
Interacting Galaxies1     
Blue compact Galaxy1     
Molecular Cloud1     

OMC science analysis processing at CAB

Around 100 OMC CCD sub-windows, each of 11x11 pixels, are downloaded to Earth about every minute, without any on-board processing. The data are received at the Integral Science Data Center, where they are completely processed in an automatic way. OMC data are then re-processed at CAB and stored in the OMC Data Server. The principal steps of this science analysis processing are the following:

  1. The information in the telemetry packages is converted to CCD pixel values.

  2. The OMC CCD sub-windows are corrected from bias and dark current. They are then convolved with the corresponding flat-field matrix.

  3. OMC CCD sub-windows obtained within periods of around 10 minutes (default value) as well as within the full Science Window are combined to obtain a better signal-to-noise ratio. A Science Window is a pointing (typically around 30 minutes) or a part of it in case of being too long (staring mode). A third solution without any combination of the OMC CCD sub-windows is also given. These three cases correspond to three different processing options.

  4. The photometric value for each source of interest is obtained by applying three different extraction masks of 1x1, 3x3 (default value) and 5x5 pixels to the combined images (see column description below). The local background, as measured on a corona around the extraction mask, is subtracted. If Source coordinates (default) is selected as the centroid method in the fill-in form, the position of the extraction mask is re-centred on the source coordinates given in the OMC Input Catalogue, allowing a maximum offset of 10 arcsec (parameter IMA_maxWcsOff in OMC OSA). Instead if Brightest pixel is selected as the centroid method, the position of the extraction mask is re-centered on the brightest pixel within the central 5x5 pixel section. This extraction software can also be run offline to modify the parameters, in order to minimize the effects of background stars when needed.

  5. The photometric values are stored as V magnitudes, thus compiling light curves with three time binnings:

    • Shot-by-Shot: one photometric point per shot.
    • 10 minutes: around one photometric point each 10 minutes. Shots with exposure smaller than 20 seconds are rejected to increase the signal-to-noise ratio.
    • ScW-by-ScW: one photometric point per Science Window. Shots with exposure smaller than 60 seconds are rejected to increase the signal-to-noise ratio.

    We must notice that due to the rejection of the shortest shots when using time binnings of 10 minutes and ScW-by-ScW, the brightest sources will not be processed when selecting these time binnings, because they will be saturated in the remaining shots. For these sources, the user should select the time binning of Shot-by-Shot, and then filter the results to keep only the shortest shots with no saturation effects.

In this science analysis, OMC Input Catalogue Version V0005 and the following versions of OMC OSA 7.0, OSA 9.0 and OSA 10.1 (Off-line Science Analysis) components were used:

OSA 7.0 (revolutions 11-890):

omc_science_analysis 5.3.1
  omc_scw_analysis 5.3.1
    o_cor_science 5.3.1
      o_cor_box_fluxes 6.5.1
    o_gti 5.3.1
      gti_create 2.2
      gti_attitude 1.5
      gti_merge 1.6
    o_src_analysis 5.3.1
      o_src_get_fluxes 8.1
      o_src_compute_mag 4.4
  omc_obs_analysis 5.3.1
    o_src_collect 2.4

OSA 9.0 (revolutions 891-1280):

omc_science_analysis 6.0
  omc_scw_analysis 6.0
    o_cor_science 6.0
      o_cor_box_fluxes 6.5.1
    o_gti 6.0
      gti_create 2.3
      gti_attitude 1.5
      gti_merge 1.6
    o_src_analysis 6.0
      o_src_get_fluxes 8.2
      o_src_compute_mag 4.4
  omc_obs_analysis 6.0
    o_src_collect 2.4

OSA 10.1 (revolutions 1281-->):

omc_science_analysis 6.0.2
  omc_scw_analysis 6.0.2
    o_cor_science 6.0.2
      o_cor_box_fluxes 6.5.1
    o_gti 6.0.2
      gti_create 2.3
      gti_attitude 1.5
      gti_merge 1.6
    o_src_analysis 6.0.2
      o_src_get_fluxes 8.3
      o_src_compute_mag 4.4
  omc_obs_analysis 6.0.2
    o_src_collect 2.4

Best photometric accuracy achievable

Expected best photometric accuracy in V magnitudes of the OMC for different V values and various effective integration times. A typical observation totals 300 s of effective integration time.

V magnitude
Eff. integration (s) 8 10 12 14 16
10       0.007 0.02 0.1 -- --
300       -- 0.005 0.01 0.04 0.3
900       -- 0.003 0.006 0.026 0.17

The above values do not account for possible systematic biases.


Calibration status

In August 2006 a new set of flatfield matrices (Version 5) as well as photometric calibration files (Version 5) were delivered by the OMC team to ISDC. These new IC (Instrument Characteristic) files improved substantially the old ones (Version 4).


The data currently available at the OMC Data Server were analyzed using the new IC files. In the following table the version number of each IC file used in the analysis is given:

IC Data Structure Version
OMC.-PHOT-CAL 5
OMC.-FLAT-CAL 5
OMC.-BDPX-CAL 2
OMC.-DARK-CAL 2
OMC.-GOOD-LIM 8
INTL-GOOD-LIM 2

Column description of output fits files (OMC light curves)

REVOL Revolution number valid for time of data taking
SWID Science Window identifier from which this row was taken
TFIRST Time of the first data element in ISDC Julian days
BARYTIME Barycentric time for the first data element in ISDC Julian days
TELAPSE Elapsed time of the integration in seconds
EXPOSURE Effective integration time in seconds
SHOTTYPE Type of shots used for building integration (1=Photometric, 2=Science)
OMC_ID OMC catalogue source identifier (IOMC number)
TYPE_TAR Target type (1=Photometric, 2=Science, 3=Dark Current)
RA_OBJ Source right ascension in degrees
DEC_OBJ Source declination in degrees
FLUX_1 Flux in electron/s derived from 1x1 integration boxes
ERFLUX_1 Error estimate for FLUX_1
FLUX_3 Flux in electron/s derived from 3x3 integration boxes
ERFLUX_3 Error estimate for FLUX_3
FLUX_5 Flux in electron/s derived from 5x5 integration boxes
ERFLUX_5 Error estimate for FLUX_5
SKYBACK Mean flux from sky background in electron/pixel/s
SKYERROR Error estimate for SKYBACK
SIZE_MAG Integration box size for deriving MAG_V
MAG_V Computed V (Johnson) magnitude (default value)
ERRMAG_V Error estimate for V magnitude
CATMAG_V Catalog V (Johnson) magnitude
CATERR_V Catalog error estimate for V magnitude
MAG_V1 Computed V magnitude for the 1x1 pixel area
ERMAG_V1 Error estimate for V magnitude in 1x1 pixel area
MAG_V3 Computed V magnitude for the 3x3 pixel area
ERMAG_V3 Error estimate for V magnitude in 3x3 pixel area
MAG_V5 Computed V magnitude for the 5x5 pixel area
ERMAG_V5 Error estimate for V magnitude in 5x5 pixel area
PROBLEMS Flag for various problems (0 = no problems)
NOISE_LL Read-out noise in e- (low gain, left ROP)
NOISE_LR Read-out noise in e- (low gain, right ROP)
NOISE_HL Read-out noise in e- (high gain, left ROP)
NOISE_HR Read-out noise in e- (high gain, right ROP)
CENTRING_X Derived X-axis offset of the source from the box centre (pixels)
CENTRING_Y Derived Y-axis offset of the source from the box centre (pixels)
PSF_FWHM Effective PSF FWHM in pixels
X_TAR X coordinate of the lower left pixel of the target box (pixels)
Y_TAR Y coordinate of the lower left pixel of the target box (pixels)
RANK On-board sequence number of box
RA_FIN Derived right ascension in degrees
RA_FIN_ERR Standard error for RA_FIN*cos(DEC_FIN)
DEC_FIN Derived declination in degrees
DEC_FIN_ERR Standard error for DEC_FIN

PROBLEMS column description

This is the meaning of PROBLEMS column in output fits files. Problems are stored in an unsigned integer register, and any problem encountered is logically ANDed to the existing register value. Deconstruction of the total into its only possible component values reveal the individual PROBLEMS. Problem values 64 and 2048 are reserved for future improvements.

NAME VALUE   DESCRIPTION
OMC_PROBLEM_NONE 0   No problems
OMC_PROBLEM_CENTROID_OUT 1   Centroid does not match maxWcsOff radius; The WCS position has been used as centroid
OMC_PROBLEM_EXTRAPOLATED_MAG 2   The magnitude was extrapolated
OMC_PROBLEM_BAD_CENTROID 4   No centroid is available or is inaccurate
OMC_PROBLEM_BAD_PSF 8   Bad PSF. A default value was used
OMC_PROBLEM_ANOMALOUS_PSF 16   The PSF shape is anomalous
OMC_PROBLEM_LOW_FLUX_1 32   Flux of central pixel too low
OMC_PROBLEM_BADPIXEL_SKY 128   Bad pixel found in sky background
OMC_PROBLEM_BADPIXEL_RIM_5 256   Bad pixel found in 5x5 rim
OMC_PROBLEM_BADPIXEL_RIM_3 512   Bad pixel found in 3x3 rim
OMC_PROBLEM_BADPIXEL_RIM_1 1024   Central pixel bad
OMC_PROBLEM_SKY_ERROR 4096   Sky error larger than accepted limit
OMC_PROBLEM_UNKNOWN_MAG 8192   Magnitude could not be calculated
OMC_PROBLEM_EXTND_SRC 16384   Source is extended - flux not valid
OMC_PROBLEM_COORD_OUT 32768   WCS position close to the edge or out of the OMC box; The brightest pixel has been used.

Known limitations


Other foreseen functionalities

A variety of data analysis tools are being developed and will be available in the near future. The aim of these tools is to provide added-value functionalities to the system giving the ability to perform data analysis tasks remotely.


Retrieval of OMC images: The user will be able to get raw and corrected images. In the last case the OMC sub-windows will be corrected for bias, dark current and flatfield.


Time series analysis: The operation of INTEGRAL from a high orbit allows a continuous observation (only interrupted by the radiation belts crossing) for periods of several weeks giving a unique photometric capability that cannot be addressed from ground-based observatories. Our aim is that the OMC data server also provides information in the frequency domain. Given the diversity of the OMC targets (AGNs, X-rays binaries, gamma-ray burst, eclipsing binaries, pulsating variables, ...) and the variability patterns (long/short-term variations, mono/multi-periodicity,...), a detailed study on the techniques to be applied in each case must be performed.


Data mining. Light curve characterization: Even though the OMC data server allows making queries by object type based on the classification provided by SIMBAD, it is clear that this classification can be greatly improved with the use of the OMC data. Given the vast amount of data to be handled, classification procedures based on the visual inspection by experts are not adequate and data mining techniques must be used instead. We are presently working in a neural network system to classify light curves of periodic variable stars. The network will be trained with the HIPPARCOS light curves and, in a first step, it will allow for the automatic classification of eclipsing binary stars with a further extension to other types of variable stars (e.g. pulsating variables). The system is designed to perform a unsupervised topological mapping based on morphological proximity among the light curves.




Version 3.0 - March 2012     © CAB (INTA-CSIC) Home