issue with EOVSA synoptic imaging pipeline #287
Description
EOVSA Synoptic Pipeline Update
The new EOVSA synoptic pipeline, version 2.0, is now available for testing. Below are the key details to help you get started:
How to Run the Pipeline
-
Command to execute the pipeline for a specific date (example for 2021 Nov 24):
/bin/bash /common/python/eovsapy-src/eovsapy/shellScript/pipeline.sh --date 2021-11-24 --ncpu 1 --version v2.0
-
Output Location:
The output FITS files are stored at/data1/eovsa/fits/synoptic/2021/11/24/. They are also accessible via the web at https://ovsa.njit.edu/fits/synoptic/2021/11/24/. -
File Naming Convention:
Files follow a new naming convention to differentiate between types:eovsa.synoptic_daily.20211124T200000_UTC.s00-01.tb.fits eovsa.synoptic_60m.20211124T170000_UTC.s00-01.tb.fitsNote: The daily synoptic image aggregates all the 60-minute synoptic images from that day.
-
Important: Ensure you specify
--version v2.0when using the new code. Using the old version (--version v1.0) or omitting the version will default to the original pipeline. -
Local Testing:
You can also test the pipeline locally using a UDBms file as input:python suncasa/eovsa/eovsa_synoptic_imaging_pipeline.py UDB20211124.ms
The results will be stored in your current directory.
## Current Issues and Developments
-
Rotation Timing for 60-minute Files:
Theeovsa.synoptic_60mimages currently align to 20 UT. We plan to adjust this to better reflect actual observation times. -
Multi-threading on Linux:
The code is designed to leverage multiple CPU cores for parallel processing, which effectively splits the daily UDBms into multiple chunks to be processed independently across different cores. This feature works seamlessly on macOS. The runtime of one UBDms file is less than an hour. However, challenges arise when running the code on our pipeline server under Linux, where the process tends to hang. This issue may be related to how CASA handles file operations in a multiprocessing environment. As a temporary measure, please set the number of CPU threads to 1. The typical run time is 8 hours. I will explore this issue further and seek insights from how the ovro-lwa-solar pipeline handles multiprocessing, which could offer a solution. I took a peek at what you guys have implemented in the ovro code, and didn't find out why the EOVSA code failed. -
Initial Imaging Model:
The pipeline uses hourly image models for self-calibration, which may be unreliable during high flare activity or with poor instrumental calibration. A robust check is required to ensure a good initial model is used.