write_filex() function cuts levels in management subsections to two digits while dssat allows three digit levels

[gonzarizzo]

Hello Phil,

I have found an bug in the write_filex() function. The problem is as follows: when you try to create a filex with more than 99 levels of a treatment, the write function cuts the treatment levels to two digits.

In the example below, I modified the last treatment of a WHX file to set a cultivar treatment to 999. While the functions writes the level 999 in the treatment's level section of the filex, it cuts the level 999 to 99 in the CULTIVARS section of the filex. I think this problem has to do with the column width set in the function. However, note that DSSAT allows to have 3 digits levels in the CULTIVARS section (for example).

library(tidyverse)
library(DSSAT)
options(DSSAT.CSM = "/DSSAT48/dscsm048")
file.copy("/DSSAT48/Wheat/KSAS8101.WHX", ".")
fileX <- read_filex("KSAS8101.WHX")
fileX$`TREATMENTS                        -------------FACTOR LEVELS------------` <-
  fileX$`TREATMENTS                        -------------FACTOR LEVELS------------` %>%
  mutate(CU = ifelse(N == 6, 999, CU))

fileX$CULTIVARS <-
  fileX$CULTIVARS %>%
  mutate(C = list(c(1, 999)))

# write the fileX
write_filex(fileX, "KSAS8102.WHX")

# run the treatment six
run_dssat("C", "KSAS8102.WHX 6")  # it won't run unless you manually add the missing 9 in line 29 of the created fileX

The result:

How the right fileX should look like:

[palderman]

Hi @gonzarizzo,

Have you confirmed that it's running the correct cultivar number? I believe that the two-digit limitation is based on the DSSAT-CSM code. I can pull up the Fortran code to be sure.

[gonzarizzo]

I have double checked and it runs. I have edited the original question adding two lines to write and run the fileX.

Thank you for your prompt response.
Gonzalo

[palderman]

The three digit treatment number might depend on the run mode. If the run mode is 'Q' for sequence simulations, then it only seems to read the first two digits of the N column under the *TREATMENTS section:

https://github.com/DSSAT/dssat-csm-os/blob/1fd74befa93f05407bca4d8b87c89993ee7d905c/InputModule/ipexp.for#L294-L302

https://github.com/DSSAT/dssat-csm-os/blob/1fd74befa93f05407bca4d8b87c89993ee7d905c/InputModule/ipexp.for#L865C1-L866

The CU column does seem to work okay and it seems like the *CULTIVARS section should work, too:

https://github.com/DSSAT/dssat-csm-os/blob/1fd74befa93f05407bca4d8b87c89993ee7d905c/InputModule/IPMAN.for#L858C3-L858C3

https://github.com/DSSAT/dssat-csm-os/blob/1fd74befa93f05407bca4d8b87c89993ee7d905c/InputModule/IPMAN.for#L876

The inconsistency across run modes is a bit frustrating, but I guess the three-digit format would be the best one to use as the default.

In the mean time, following on from your example above you should theoretically be able to override the default format with something like:

trt_v_fmt <- DSSAT:::filex_v_fmt("TREATMENTS")
trt_v_fmt["N"] <- "%3.0f"
attr(filex$TREATMENTS, "v_fmt") <- trt_v_fmt
cul_v_fmt <- DSSAT:::filex_v_fmt("CULTIVARS")
cul_v_fmt["C"] <- "%3.0f"
cul_v_fmt["CR"] <- "%2s"
attr(filex$CULTIVARS, "v_fmt") <- cul_v_fmt
write_filex(filex, "KSAS8102.WHX", force_std_fmt = FALSE)

I haven't tested that capability in some time, so let me know if it doesn't work.

[palderman]

@gonzarizzo, have you perchance tested to see if this is true for other sections (FERTILIZERS, FIELDS, PLANTING DETAILS etc)?

[gonzarizzo]

Yes, the same happens in Fertilizers, Fields and Planting details. And also in the Number of Treatments (first column in the first section of the FILEX).

[gonzarizzo]

With respect to the number of experiments, running MZX allows to run up to 999 experiments in one fileX. See below how it should be set:

top:

bottom:

Here I am running 100 experiments. But you have to move one digit to the left.

[palderman]

@gonzarizzo, Thanks for checking on these other sections. By the way, in my previous example I failed to adjust the column width for R. The amended example would look something like:

trt_v_fmt <- DSSAT:::filex_v_fmt("TREATMENTS")
trt_v_fmt["N"] <- "%3.0f"
trt_v_fmt["R"] <- "%1.0f"
attr(filex$TREATMENTS, "v_fmt") <- trt_v_fmt
cul_v_fmt <- DSSAT:::filex_v_fmt("CULTIVARS")
cul_v_fmt["C"] <- "%3.0f"
cul_v_fmt["CR"] <- "%2s"
attr(filex$CULTIVARS, "v_fmt") <- cul_v_fmt
write_filex(filex, "KSAS8102.WHX", force_std_fmt = FALSE)

[gonzarizzo]

Thank you, @palderman!

BTW: Are you planning to include these changes in any new package version release? And also, would you be willing to make any new release version available via conda-forge?

[gonzarizzo]

@gonzarizzo, Thanks for checking on these other sections. By the way, in my previous example I failed to adjust the column width for R. The amended example would look something like:

trt_v_fmt <- DSSAT:::filex_v_fmt("TREATMENTS")
trt_v_fmt["N"] <- "%3.0f"
trt_v_fmt["R"] <- "%1.0f"
attr(filex$TREATMENTS, "v_fmt") <- trt_v_fmt
cul_v_fmt <- DSSAT:::filex_v_fmt("CULTIVARS")
cul_v_fmt["C"] <- "%3.0f"
cul_v_fmt["CR"] <- "%2s"
attr(filex$CULTIVARS, "v_fmt") <- cul_v_fmt
write_filex(filex, "KSAS8102.WHX", force_std_fmt = FALSE)

This solution doesn't run:

> trt_v_fmt <- DSSAT:::filex_v_fmt("TREATMENTS")
Error: object 'filex_v_fmt' not found

[palderman]

@gonzarizzo, you indicated by email that the filex_v_fmt() isn't working for some reason. You could try the following instead:

trt_v_fmt <- attr(filex$TREATMENTS, "v_fmt")
trt_v_fmt["N"] <- "%3.0f"
trt_v_fmt["R"] <- "%1.0f"
attr(filex$TREATMENTS, "v_fmt") <- trt_v_fmt
cul_v_fmt <- attr(filex$CULTIVARS, "v_fmt")
cul_v_fmt["C"] <- "%3.0f"
cul_v_fmt["CR"] <- "%2s"
attr(filex$CULTIVARS, "v_fmt") <- cul_v_fmt
write_filex(filex, "KSAS8102.WHX", force_std_fmt = FALSE)

[gonzarizzo]

@palderman I have added your code to the example but still it does not work. See below the error it produces:

# load packages and filex
library(tidyverse)
library(DSSAT)
options(DSSAT.CSM = "/DSSAT48/dscsm048")
file.copy("/DSSAT48/Wheat/KSAS8101.WHX", ".")
fileX <- read_filex("KSAS8101.WHX")
fileX$`TREATMENTS                        -------------FACTOR LEVELS------------` <-
  fileX$`TREATMENTS                        -------------FACTOR LEVELS------------` %>%
  mutate(CU = ifelse(N == 6, 999, CU))

# modify filex
fileX$CULTIVARS <-
  fileX$CULTIVARS %>%
  mutate(C = list(c(1, 999)))

# Phil's solution
trt_v_fmt <- attr(fileX$TREATMENTS, "v_fmt")
trt_v_fmt["N"] <- "%3.0f"
trt_v_fmt["R"] <- "%1.0f"
attr(fileX$TREATMENTS, "v_fmt") <- trt_v_fmt
cul_v_fmt <- attr(fileX$CULTIVARS, "v_fmt")
cul_v_fmt["C"] <- "%3.0f"
cul_v_fmt["CR"] <- "%2s"
attr(fileX$CULTIVARS, "v_fmt") <- cul_v_fmt
write_filex(fileX, "KSAS8102.WHX", force_std_fmt = FALSE)

Output:

Error in `map()`:
ℹ In index: 1.
Caused by error in `write_tier()`:
! Tier data has no v_fmt attribute.
Run `rlang::last_trace()` to see where the error occurred.

[gonzarizzo]

@palderman it seems that the problem is that there is no attribute "v_fmt", indeed attr(fileX, "v_fmt") returns NULL.

[palderman]

A v_fmt should be within each element of fileX. Try e.g. attr(fileX$CULTIVARS, "v_fmt")

[gonzarizzo]

Yes, it returns NULL.

[palderman]

That explains why my second suggestion was not working. Could you try reinstalling from the develop branch of the repository (remotes::install_github("palderman/DSSAT", ref = "develop")) and then trying my first suggestion again?