README.md

---
tags:
- Instance Segmentation
- Vision Transformers
- CNN
- Optical Space Missions
- Artefact Mapping
- ESA

pretty_name: XAMI-model
license: mit
datasets:
- iulia-elisa/XAMI-dataset
---

<div align="center">
<h1> XAMI-model: XMM-Newton optical Artefact Mapping for astronomical Instance segmentation </h1>
</div>

This repository contains the weights of the **[XAMI model](https://github.com/ESA-Datalabs/XAMI-model)**. The model is trained on images from the XAMI dataset (available on Github and HuggingFace). The images are astronomical observations from the Optical Monitor (XMM-OM) onboard the XMM-Newton X-ray mission of the European Space Agency (ESA).

Information about the XMM-OM can be found here: 

- [The ESA website](https://www.cosmos.esa.int/web/xmm-newton/technical-details-om)
- [The article *The XMM-Newton optical/UV monitor telescope*](https://ui.adsabs.harvard.edu/abs/2001A%26A...365L..36M/abstract).

## Cloning the repository

```bash
git clone https://github.com/ESA-Datalabs/XAMI-model.git
cd XAMI-model

# creating the environment
conda env create -f environment.yaml
conda activate xami_model_env

# Install the package in editable mode
pip install -e .
```

## Downloading the dataset and model checkpoints from HuggingFace

The dataset is splited into train and validation categories and contains annotated artefacts in COCO format for Instance Segmentation. We use multilabel Stratified K-fold (k=4) to balance class distributions across splits. We choose to work with a single dataset splits version (out of 4) but also provide means to work with all 4 versions.

The [Dataset-Structure.md](https://github.com/ESA-Datalabs/XAMI-dataset/blob/main/Datasets-Structure.md) offers more details about the dataset structure. We provide the following dataset formats: COCO format for Instance Segmentation (commonly used by [Detectron2](https://github.com/facebookresearch/detectron2) models) and YOLOv8-Seg format used by [ultralytics](https://github.com/ultralytics/ultralytics).

<!-- 1. **Downloading** the dataset archive from [HuggingFace](https://huggingface.co/datasets/iulia-elisa/XAMI-dataset/blob/main/xami_dataset.zip).

```bash
DEST_DIR='.' # destination folder for the dataset (should usually be set to current directory)

huggingface-cli download iulia-elisa/XAMI-dataset xami_dataset.zip --repo-type dataset --local-dir "$DEST_DIR" && unzip "$DEST_DIR/xami_dataset.zip" -d "$DEST_DIR" && rm "$DEST_DIR/xami_dataset.zip"
``` -->


Check the [dataset_and_model.ipynb](https://github.com/ESA-Datalabs/XAMI-model/blob/main/dataset_and_model.ipynb) for downloading the dataset and model weights.

## Model Inference

After cloning the repository and setting up the environment, use the following code for model loading and inference:

```python
from xami_model.inference.xami_inference import InferXami

det_type = 'rtdetr' # 'rtdetr'  'yolov8'

detr_checkpoint = f'./xami_model/train/weights/{det_type}_sam_weights/{det_type}_detect_300e_best.pt'
sam_checkpoint = f'./xami_model/train/weights/{det_type}_sam_weights/{det_type}_sam.pth'

detr_sam_pipeline = InferXami(
    device='cuda:0',
    detr_checkpoint=detr_checkpoint,
    sam_checkpoint=sam_checkpoint,
    model_type='vit_t', # the SAM checkpoint and model_type (vit_h, vit_t, etc.) must be compatible
    use_detr_masks=True,
    detr_type=det_type)

masks = detr_sam_pipeline.run_predict('./example_images/S0893811101_M.png', show_masks=True)
```

For training the model, check the training [README.md](https://github.com/ESA-Datalabs/XAMI-model/blob/main/xami_model/train/README.md).

## © Licence 

This project is licensed under [MIT license](LICENSE).