CCSID conversion issue in SequentialFile

[rgala]

It looks like during reading/writing records from source physical files using SequentialFile class, the CCSID of field is ignored and data is interpreted in job's CCSID. In my case, after connecting to AS400.RECORDACCESS service, the job QRWTSRVR that is handling the connection has CCSID set to 278 (I have CCSID 278 set on my profile). The source file has CCSID of 37. Variant characters are incorrectly converted.

Steps to reproduce:

1. Temporarily set your profiles CCSID to 278 by running CHGPRF CCSID(278)
2. CRTSRCPF FILE(SOMELIB/TEST)
3. CHGPF FILE(SOMELIB/TEST) CCSID(37)
4. ADDPFM FILE(SOMELIB/TEST) MBR(TEST) SRCTYPE(RPGLE)
5. Put some variant characters to a member, like [^]\, for example.
6. Read records in Java with something like below:

AS400 as400 = new AS400();
as400.connectService(AS400.RECORDACCESS);
SequentialFile sf = new SequentialFile(as400, "/QSYS.LIB/SOMELIB.LIB/TEST.FILE/TEST.MBR");
sf.setRecordFormat();
Record[] records = sff.readAll();
for (Record record : recordss) {
    System.out.println(record.getField(2));
}

What will be printed is §¬¤É instead of [^]\

When debugging, the as400Data_ field of record variable contains 0xB5 0x5F 0X9F 0x71 (starting from index 12) which are 278 equivalents of [^]\, in CCSID 37 it's §¬¤É and this is what is converted to a Java string.

I tried to set CCSID to 37 on AS400 class object (as400.setCcsid(37)) before connecting, but it did not help. The workaround I found was to alter record format and force CCSID 278 on SRCDTA field, but I don't like this solution at all.

RecordFormat rf = sf.getRecordFormat();
AS400Text textField = new AS400Text(rf.getFieldDescriptions()[2].getLength(), 278);
RecordFormat format = new RecordFormat(rf.getName());
format.addFieldDescription(rf.getFieldDescription(0));
format.addFieldDescription(rf.getFieldDescription(1));
format.addFieldDescription(new CharacterFieldDescription(textField, rf.getFieldDescriptions()[2].getFieldName()));
sf.setRecordFormat(format);

[jeber-ibm]

Can your double check the contents of the file on the IBM i? It could be that the tool used to insert the variant characters into the file inserted the wrong characters. In my case I used STRSEU from my emulator session when my job CCSID was 278.

You can use the QSH od tool to see the bytes in the file.
For example, this is what I see when I look at the file that I edited using STRSEU

od -tx -C /qsys.lib/somelib.lib/test.file/test.mbr
/qsys.lib/somelib.lib/test.file/test.mbr CCSID = 37
0000000 40404040 405f4a4f 71404040 40404040
0000020 40404040 40404040 40404040 40404040

5f4a4f71 in CCSID 37 is not the CCSID 37 representation of [^]\ (which would be BAB0BBE0)

values hex(cast('[^]\' as CHAR(4) CCSID 37)) 00001 BAB0BBE0

I'm not sure how my emulator messed the values up so badly as the CCSID 278 interpretation of [^]\ is (B55F9F71)
values hex(cast('[^]\' as CHAR(4) CCSID 278)) 00001 B55F9F71

[rgala]

The file content is CCSID 37:

OD output:

/qsys.lib/somelib.lib/test.file/test.mbr CCSID = 37
0000000 bab0bbe0 40404040 40404040 40404040
0000020 40404040 40404040 40404040 40404040
*
0000120

DSPPFM:

F0F0F0F1 F0F0F2F5 F0F9F2F9 BAB0BBE0 40404040 40404040 40404040 40404040 40404040