diff --git a/README.md b/README.md index 5b7ac3c6..53acc0de 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ **Please refer to the [auto_rx wiki](https://github.com/projecthorus/radiosonde_auto_rx/wiki) for the latest information.** -This project is built around [rs1279's RS](https://github.com/rs1729/RS) demodulators, and provides a set of utilities ('auto_rx') to allow automatic reception and uploading of [Radiosonde](https://en.wikipedia.org/wiki/Radiosonde) positions to multiple services, including: +This project is built around [rs1729's RS](https://github.com/rs1729/RS) demodulators, and provides a set of utilities ('auto_rx') to allow automatic reception and uploading of [Radiosonde](https://en.wikipedia.org/wiki/Radiosonde) positions to multiple services, including: * The [SondeHub Radiosonde Tracker](https://tracker.sondehub.org) - a tracking website specifically designed for tracking radiosondes! * APRS-IS, for display on sites such as [radiosondy.info](https://radiosondy.info). (Note that aprs.fi now blocks radiosonde traffic.) @@ -51,7 +51,7 @@ We also have a channel in the SondeHub Discord server: https://sondehub.org/go/d * [Michaela Wheeler](https://github.com/TheSkorm) - radiosonde@michaela.lgbt ## Licensing Information -All software within this repository is licensed under the GNU General Public License v3. Refer this repositories LICENSE file for the full license text. +All software within this repository is licensed under the GNU General Public License v3. Refer to this repository's LICENSE file for the full license text. Radiosonde telemetry data captured via this software and uploaded into the [Sondehub](https://sondehub.org/) Database system is licensed under [Creative Commons BY-SA v2.0](https://creativecommons.org/licenses/by-sa/2.0/). Telemetry data uploaded into the APRS-IS network is generally considered to be released into the public domain. diff --git a/auto_rx/test/README.md b/auto_rx/test/README.md index f4447b14..f35dd0fc 100644 --- a/auto_rx/test/README.md +++ b/auto_rx/test/README.md @@ -26,7 +26,7 @@ This script generates a set of low-SNR samples based on the base high-SNR sample Calibrated-level noise is added to the sample to produce a file with a user-defined Eb/No ('SNR per-bit'). If everything works 'perfectly', we should expect all the different modems to have similar PER vs Eb/No performance. However, real-world factors such as packet length, transmitter deviation, filter widths, etc will mess this up. -I wouldn't try and make too many comparions of the performance between different sonde demodulators. Better to strike +I wouldn't try and make too many comparisons of the performance between different sonde demodulators. Better to strike a baseline of current performance, and then try and improve on it. The level of noise to add is determined based on the variance of the sample. Some checking of Eb/No of generated @@ -43,13 +43,13 @@ $ python generate_lowsnr.py Notes: - * I suspect the variance measurement for the m10 sample is off. Its performing suspiciously better than the other sondes. + * I suspect the variance measurement for the m10 sample is off. It's performing suspiciously better than the other sondes. ## test_demod.py -This script run the generated samples above through different demodulation chains. +This script runs the generated samples above through different demodulation chains. -Check the processing_type dict in the script for the differnet demodulation options. +Check the processing_type dict in the script for the different demodulation options. Example: ``` diff --git a/auto_rx/test/notes/2019-03-01_detector_change.md b/auto_rx/test/notes/2019-03-01_detector_change.md index cc10af11..415ca4b8 100644 --- a/auto_rx/test/notes/2019-03-01_detector_change.md +++ b/auto_rx/test/notes/2019-03-01_detector_change.md @@ -81,7 +81,7 @@ We can see that while M10 detection performance suffered no degradation, the det My apologies to those users that may have been affected by this performance degradation! This really shows the value of having a repeatable performance testing system. -## dfm_detect Performance +## dft_detect Performance The following shows the detection performance when rs_detect is replaced with dft_detect. dft_detect has a set of internal thresholds, which are intended to prevent false-positives and mis-identification of radiosonde types. A discussion on how these thresholds have been optimized is available in [this report](./2019-03-01_dft_detect_optimization.md). **dft_detect + rtl_fm 22 kHz sample rate** @@ -119,7 +119,7 @@ SNR | RS41 | RS92 | DFM | M10 19 | RS41 | RS92 | DFM | M10 19.5 | RS41 | RS92 | DFM | M10 -We can clearly see an improvement in detection performance on all radiosonde types. The M10 results are still questionable, though as mentioned previously the very good detection performanance is likely due to the shorter header size. +We can clearly see an improvement in detection performance on all radiosonde types. The M10 results are still questionable, though as mentioned previously the very good detection performance is likely due to the shorter header size. ## Caveats and Other Issues diff --git a/auto_rx/test/notes/2019-03-01_dft_detect_optimization.md b/auto_rx/test/notes/2019-03-01_dft_detect_optimization.md index 4fa0465b..830967bf 100644 --- a/auto_rx/test/notes/2019-03-01_dft_detect_optimization.md +++ b/auto_rx/test/notes/2019-03-01_dft_detect_optimization.md @@ -1,4 +1,4 @@ -# dft_detect Threshold Optimzation +# dft_detect Threshold Optimization Mark Jessop - 2019-03-02 @@ -23,7 +23,7 @@ dft_detect was run over a set of noise samples (noise1_96k_float.bin, noise2_96k The worst-case (highest) correlation scores out of the three samples are shown for each radiosonde type: -Sonde Type | Worse-Case Correlation Score +Sonde Type | Worst-Case Correlation Score -----------|----------------------------- DFM | 0.3050 RS41 | 0.3261 diff --git a/auto_rx/test/notes/2019-03-02_performance_baseline.md b/auto_rx/test/notes/2019-03-02_performance_baseline.md index 08a67f32..04870c2b 100644 --- a/auto_rx/test/notes/2019-03-02_performance_baseline.md +++ b/auto_rx/test/notes/2019-03-02_performance_baseline.md @@ -22,6 +22,6 @@ The traces for the error-corrected decoders (RS41, RS92, DFM), exhibit the typic A few things to look into: * This is a plot of *packet*-error-rate. Can we get a measure of the number of bits in a 'packet' (whatever that may be for each sonde; it may mean multiple frames) to be able to estimate a *bit*-error-rate? How does this compare to theoretical FSK modem performance? -* Why is ths RS92 'cliff' offset from the others? Are the packets bigger? Is there something sub-optimal in the demodulation chain? +* Why is the RS92 'cliff' offset from the others? Are the packets bigger? Is there something sub-optimal in the demodulation chain? * Further investigation into what the rtl_fm samples rates mean for pre-FM-demod filter bandwidth is required, to ensure they are set optimally. * Double and triple-checking of the Eb/N0 calculations in `generate_lowsnr.py`, to be sure we're generating our low-Eb/N0 samples correctly. \ No newline at end of file diff --git a/auto_rx/test/notes/2019-03-03_generate_lowsnr_validation.md b/auto_rx/test/notes/2019-03-03_generate_lowsnr_validation.md index c6610142..adf6bcc6 100644 --- a/auto_rx/test/notes/2019-03-03_generate_lowsnr_validation.md +++ b/auto_rx/test/notes/2019-03-03_generate_lowsnr_validation.md @@ -2,18 +2,18 @@ All of the performance testing scripts in use rely on samples with 'calibrated' Signal-to-noise ratio (SNR) - we add noise to a 'golden' sample to produce a sample with a known SNR. -However, instead of using SNR in the signal bandwidth, we use [Eb/N0](https://en.wikipedia.org/wiki/Eb/N0) (SNR-per-bit). This normalises the SNR, and allows comparison between modems operating at different baud rates. Eventually, this will also allow comparison of the modems with the theoretical acheivable performance of a FSK modem. +However, instead of using SNR in the signal bandwidth, we use [Eb/N0](https://en.wikipedia.org/wiki/Eb/N0) (SNR-per-bit). This normalises the SNR, and allows comparison between modems operating at different baud rates. Eventually, this will also allow comparison of the modems with the theoretical achievable performance of a FSK modem. The calibrated SNR samples are generated using [generate_lowsnr.py](../generate_lowsnr.py). A set of [golden samples](http://rfhead.net/sondes/sonde_samples.tar.gz) (one per radiosonde type) have very high SNRs, usually about 40dB or so - enough such that all packets are easily decoded. For each sample, we measure the signal power in the sample. Then, using a calculation based on the baud rate, the sample rate, the number of bits-per-symbol, and the desired SNR, we can generate white noise with a given noise power. This is added to the sample, which is then normalised (to +/-1.0) and saved. These samples can then be run through the various demodulation chains to measure their performance. -To be sure these measurements are meainingful, we need some confidence that the right amount of noise is being applied. We can do this by generating a FSK signal with known bits, and running it through the `generate_lowsnr.py` script. The 'noisy' signals can then be passed back into a FSK demodulator, and the Bit-Error-Eate (BER) measured. [David Rowe's FSK modem](http://svn.code.sf.net/p/freetel/code/codec2-dev/README_fsk.txt) has previously been demonstrated to have performance essentially equal to that of a theoretical non-coherent FSK modem, and so is ideal for this purpose. +To be sure these measurements are meaningful, we need some confidence that the right amount of noise is being applied. We can do this by generating a FSK signal with known bits, and running it through the `generate_lowsnr.py` script. The 'noisy' signals can then be passed back into a FSK demodulator, and the Bit-Error-Rate (BER) measured. [David Rowe's FSK modem](http://svn.code.sf.net/p/freetel/code/codec2-dev/README_fsk.txt) has previously been demonstrated to have performance essentially equal to that of a theoretical non-coherent FSK modem, and so is ideal for this purpose. ## Generation of FSK The codec2-dev repository (which contains the fsk modem mentioned above) has a suite of utilities for testing modem performance. For our testing we will use the following utilities: * **fsk_get_test_bits**: Generate a sequence of test bits, based off a known pseudorandom seed. -* **fsk_mod**: Generate a FSK signal with provided baud rate, sample rate, centre frequency, and shift. Samples are accepted as one-byte-per-bit via stdin. fsk_mod usually generated real-valued outputs, but with a small modification can produce a complex output. -* **fsk_demod**: Demodulate a FSK signal, with provided sample rate and baud rate. fsk_demod can accept real and complex-valued samples - we are using compex-valued samples. +* **fsk_mod**: Generate a FSK signal with provided baud rate, sample rate, centre frequency, and shift. Samples are accepted as one-byte-per-bit via stdin. fsk_mod usually generates real-valued outputs, but with a small modification can produce a complex output. +* **fsk_demod**: Demodulate a FSK signal, with provided sample rate and baud rate. fsk_demod can accept real and complex-valued samples - we are using complex-valued samples. * **fsk_put_test_bits**: Receive the sequence of test bits, and provide BER statistics. These utilities can be chained together using bash pipes, with a basic example being: diff --git a/auto_rx/test/notes/2019-04-23_rs41_highpass.md b/auto_rx/test/notes/2019-04-23_rs41_highpass.md index 31d3bd88..04599271 100644 --- a/auto_rx/test/notes/2019-04-23_rs41_highpass.md +++ b/auto_rx/test/notes/2019-04-23_rs41_highpass.md @@ -29,7 +29,7 @@ We can see that the non-highpass decode chain (the current decode chain as of 1. ## Discussion -The RTLSDR type recommended in the installation guide are those with TCXOs. These commonly come in 0.5 and 1ppm varieties, which indicates the approximate expected drift over the rated operating temperature range (often -40 to +85˚C). Asuming the SDR has been calibrated against some kind of reference ( [GSM](https://www.rtl-sdr.com/how-to-calibrate-rtl-sdr-using-kalibrate-rtl-on-linux/), [LTE](https://gist.github.com/darksidelemm/b517e6a9b821c50c170f1b9b7d65b824)), then according to the results above, drift-related performance degradation won't be an issue for these SDRs. +The RTLSDR type recommended in the installation guide are those with TCXOs. These commonly come in 0.5 and 1ppm varieties, which indicates the approximate expected drift over the rated operating temperature range (often -40 to +85˚C). Assuming the SDR has been calibrated against some kind of reference ( [GSM](https://www.rtl-sdr.com/how-to-calibrate-rtl-sdr-using-kalibrate-rtl-on-linux/), [LTE](https://gist.github.com/darksidelemm/b517e6a9b821c50c170f1b9b7d65b824)), then according to the results above, drift-related performance degradation won't be an issue for these SDRs. For SDRs *without* TCXOs temperature related drift can be significant, even just from self-heating of the SDR during regular operation. 10-20 ppm drift during warm-up is not unheard of. Looking at the above table, depending on the SDRs initial frequency offset, that kind of drift could cause significant performance degradation. Even if the high-pass filter is added to the decode chain, there is no guarantee that non-TCXO RTLSDRs will work reliably, if at all. diff --git a/auto_rx/test/notes/2019-04-26_fsk_demod.md b/auto_rx/test/notes/2019-04-26_fsk_demod.md index 2a236cd5..e14f1a5a 100644 --- a/auto_rx/test/notes/2019-04-26_fsk_demod.md +++ b/auto_rx/test/notes/2019-04-26_fsk_demod.md @@ -1,11 +1,11 @@ # 2019-04-26 Experimental fsk_demod-based Demod Chains auto_rx release 1.1.0 introduces an alternate 'experimental' demodulation chain, which provides some potential improvements over the existing FM-demod based demodulation for certain receive situations. -The new demodulation chains utilise the 'fsk_demod' demodulator from [David Rowe](https://rowetel.com/)'s [codec2](https://github.com/drowe67/codec2) repository. This FSK demodulator was developed for use in VHF/UHF digital voice applications, ans has also been used in high-altitude balloon telemetry, in particular for the Project Horus ['Wenet'](https://github.com/projecthorus/wenet) imagery payload. The modem is [well tested](https://github.com/drowe67/codec2/blob/master/README_fsk.txt), and performs within a fraction of a dB of theoretical incoherent FSK modem performance. +The new demodulation chains utilise the 'fsk_demod' demodulator from [David Rowe](https://rowetel.com/)'s [codec2](https://github.com/drowe67/codec2) repository. This FSK demodulator was developed for use in VHF/UHF digital voice applications, and has also been used in high-altitude balloon telemetry, in particular for the Project Horus ['Wenet'](https://github.com/projecthorus/wenet) imagery payload. The modem is [well tested](https://github.com/drowe67/codec2/blob/master/README_fsk.txt), and performs within a fraction of a dB of theoretical incoherent FSK modem performance. -With tuning of filter bandwidth, the existing FM-demod system has been optimized to acheive very good PER performance (possibly close to theoretical, but this needs to be verified), however where it has problems is with frequency offsets. These offsets can be due to thermal drift in the radiosonde (DFM, LMS6, and RS92-NGPs are known to drift), or drift in the receiver (due a poor-quality LO). As has been [noted previously](https://github.com/darksidelemm/radiosonde_auto_rx/blob/testing/auto_rx/test/notes/2019-04-23_rs41_highpass.md), the performance of the existing decoders falls off considerably with frequency offsets, with approx 3dB degradation at 3kHz offset. +With tuning of filter bandwidth, the existing FM-demod system has been optimized to achieve very good PER performance (possibly close to theoretical, but this needs to be verified), however where it has problems is with frequency offsets. These offsets can be due to thermal drift in the radiosonde (DFM, LMS6, and RS92-NGPs are known to drift), or drift in the receiver (due a poor-quality LO). As has been [noted previously](https://github.com/darksidelemm/radiosonde_auto_rx/blob/testing/auto_rx/test/notes/2019-04-23_rs41_highpass.md), the performance of the existing decoders falls off considerably with frequency offsets, with approx 3dB degradation at 3kHz offset. -Instead of passing the radiosonde signal through a FM demodulator and performing data-slicing, bandpass filtered and resampled complex samples are fed into fsk_demod, which performs frequency estimation, enabling it to track the radiosonde signal anywhere within the supplied passband (approx 35 kHz of bandwidth). The advantage is that ideal FSK demodulation performance is acheived so long as the signal is contained within that passband. The downside of this approach is that it is susceptible to in-band interference that is equal-to or stronger-than the wanted signal. In these situations the modem will 'lock on' to the strongest two peaks, resulting in no decode. +Instead of passing the radiosonde signal through a FM demodulator and performing data-slicing, bandpass filtered and resampled complex samples are fed into fsk_demod, which performs frequency estimation, enabling it to track the radiosonde signal anywhere within the supplied passband (approx 35 kHz of bandwidth). The advantage is that ideal FSK demodulation performance is achieved so long as the signal is contained within that passband. The downside of this approach is that it is susceptible to in-band interference that is equal-to or stronger-than the wanted signal. In these situations the modem will 'lock on' to the strongest two peaks, resulting in no decode. ## Demodulator Performance @@ -28,7 +28,7 @@ We can see there is a small amount (approx 1dB) degradation in performance betwe With a 3kHz frequency offset, the performance of the FM-demod decode chains degraded significantly, to the point at which the RS92 decoder fails completely due to the tight filtering used. No degradation is observed on the fsk_demod chain. -There is likely some optimizations that can be performed to improve performance, but in real-world decoding situations the slight performance changes between the FM and FSK demodulators are unlikely to be noticable - except in situations where frequency drift is encountered. +There is likely some optimizations that can be performed to improve performance, but in real-world decoding situations the slight performance changes between the FM and FSK demodulators are unlikely to be noticeable - except in situations where frequency drift is encountered. ## Modem Statistics diff --git a/demod/mod/README.md b/demod/mod/README.md index c82d204b..595c91b7 100644 --- a/demod/mod/README.md +++ b/demod/mod/README.md @@ -49,7 +49,7 @@ alternative decoders using cross-correlation for better header-synchronization The high modulation index has advantages in IQ-decoding.
`--ecc2` uses soft decision for 2-error words. If weak signals frequently produce errors, it is likely that more than 2 errors occur in a received word. Since there is no additional frame protection (e.g. CRC), the - frames will not be decoded reliably in weak conditions. The `--dist` option has a thredshold for the number + frames will not be decoded reliably in weak conditions. The `--dist` option has a threshold for the number of errors per packet.