XML Validation with non local schemas

Viewed 204

currently I participate in a project where I have to among other things validate XML files. The files will be uploaded by users via a REST API which is written in Java with the Spring Framwork. The schema file is located as URL[1] in the XML file [2] as attribute "schemaLocation". It can also be possible that the XSD file includes multiple other schema files. The user uploaded files are IO-Link Device Description (IODD) files.

So, my current problem is loading the included validation schemes in the main schema. It is not my goal to download the schemas and use them to validate. The whole process has to be dynamic. I also don't want to use the IODDChecker which is provided by IO-Link itself.

I read that this can be done by the ResourceResolver Interface but I cannot find any implementations to load included schemas from the main schema via URL or something like that.

So, can you help me to find a solution for that problem?

Thank you in advance!


This is the method which will validate the file:

public boolean isValid(String file) {
    if (file == null || file.isEmpty() || !Files.exists(Path.of(file)) || !Files.isReadable(Path.of(file)))
        return false;
    else if (this.getStamp() == null || this.getStamp().getChecker() == null)
        return false;
    else if (this.getStamp().getCrc().isEmpty())
        return false;

    try {
        SchemaFactory factory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);

        factory.setErrorHandler(new LineNumberErrorHandler());

        Schema schema = factory.newSchema(XsdReceiver.receive(this.schemaLocation));
        Validator validator = schema.newValidator();
        //validator.validate(new StreamSource(new ByteArrayInputStream(data)));

        validator.setResourceResolver(factory.getResourceResolver());
        validator.validate(new StreamSource(new File(file)));
    } catch (Exception e) {
        return false;
    }

    return true;
}

This is the Schema receiver method. It works but when the schema has includes then the validation process fails. (Error messages below this this code.

public static Source receive(String url) {
    url = url.contains(" ") ? url.replace(" ", "/") : url;

    try {
        URL u = new URL(url);
        HttpURLConnection c = (HttpURLConnection)u.openConnection();
        int status = c.getResponseCode();

        if (status == HttpURLConnection.HTTP_MOVED_TEMP
                || status == HttpURLConnection.HTTP_MOVED_PERM
                || status == HttpURLConnection.HTTP_SEE_OTHER
        )
            c = (HttpURLConnection) new URL(c.getHeaderField("Location")).openConnection();


        return new StreamSource(c.getInputStream());
    } catch (IOException e) {
        e.printStackTrace();
    }

    return null;
}

The error messages I received from the validator.

Line: 3) schema_reference.4: Failed to read schema document 'IODD-Primitives1.1.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
Line: 4) schema_reference.4: Failed to read schema document 'IODD-Datatypes1.1.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
Line: 5) schema_reference.4: Failed to read schema document 'IODD-Variables1.1.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
Line: 6) schema_reference.4: Failed to read schema document 'IODD-Events1.1.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
Line: 7) schema_reference.4: Failed to read schema document 'IODD-UserInterface1.1.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
Line: 8) schema_reference.4: Failed to read schema document 'IODD-Communication1.1.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
Line: 180) src-resolve: Cannot resolve the name 'DeviceIdT' to a(n) 'type definition' component.
Line: 180) src-resolve: Cannot resolve the name 'DeviceIdT' to a(n) 'simpleType definition' component.
Line: 191) src-resolve: Cannot resolve the name 'DeviceIdT' to a(n) 'type definition' component.
Line: 228) src-resolve: Cannot resolve the name 'CollectionT' to a(n) 'type definition' component.
Line: 292) src-resolve: Cannot resolve the name 'ObjectT' to a(n) 'type definition' component.
Line: 303) src-resolve: Cannot resolve the name 'CollectionT' to a(n) 'type definition' component.
Line: 312) src-resolve: Cannot resolve the name 'DataItemT' to a(n) 'type definition' component.
Line: 12) src-resolve: Cannot resolve the name 'DocumentInfoT' to a(n) 'type definition' component.
Line: 15) src-resolve: Cannot resolve the name 'CommNetworkProfileT' to a(n) 'type definition' component.
Line: 16) src-resolve: Cannot resolve the name 'ExternalTextCollectionT' to a(n) 'type definition' component.
Line: 22) src-resolve: Cannot resolve the name 'StampT' to a(n) 'type definition' component.
Line: 152) src-resolve: Cannot resolve the name 'TextRefT' to a(n) 'type definition' component.
Line: 153) src-resolve: Cannot resolve the name 'TextRefT' to a(n) 'type definition' component.
Line: 168) src-resolve: Cannot resolve the name 'TextRefT' to a(n) 'type definition' component.
Line: 169) src-resolve: Cannot resolve the name 'TextRefT' to a(n) 'type definition' component.
Line: 195) src-resolve: Cannot resolve the name 'TextRefT' to a(n) 'type definition' component.
Line: 196) src-resolve: Cannot resolve the name 'TextRefT' to a(n) 'type definition' component.
Line: 238) src-resolve: Cannot resolve the name 'DatatypeCollectionT' to a(n) 'type definition' component.
Line: 239) src-resolve: Cannot resolve the name 'VariableCollectionT' to a(n) 'type definition' component.
Line: 250) src-resolve: Cannot resolve the name 'ErrorTypeCollectionT' to a(n) 'type definition' component.
Line: 257) src-resolve: Cannot resolve the name 'EventCollectionT' to a(n) 'type definition' component.
Line: 263) src-resolve: Cannot resolve the name 'UserInterfaceT' to a(n) 'type definition' component.

[1] https://www.io-link.com/IODD/2010/10/IODD1.1.xsd
[2] https://ioddfinder.io-link.com/productvariants/search/11765 (example for IO-Link product TV7105)

1 Answers

OK, I solved the problem on my own. In short I had to create a class which implements the interface LSResourceResolver and overrides the method resolveResource.

The long answer is:
First I created the class XsdReceiver and let it implement the interface LSResourceResolver. Also the method resolveResource must be overwritten now. This method is finally responsible for querying missing XSD files and returning them as resource LSInput. (Like LSResourceResolver, LSInput is an interface and must therefore be implemented as a separate class. I had simply called it Input). Finally I used the method setResourceResolver and instantiated as parameter the created class XsdReceiver.

Functionality:
The method isValid is supposed to check the XML schema for validity, among other things. The necessary schema file can be found in the root tag. It must also be passed manually to the SchemaFactory class. After the method is called and the main schema file is read, every referenced XML or XSD file is automatically passed to the method resolveResource. The SchemaFactory class does this for us.

Have fun :D


public boolean isValid(byte[] data) {
    if (this.getStamp() == null || this.getStamp().getChecker() == null)
        return false;
    else if (this.getStamp().getCrc().isEmpty())
        return false;

    try {
        SchemaFactory factory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);

        factory.setErrorHandler(new LineNumberErrorHandler());
        factory.setResourceResolver(new XsdReceiver());

        Schema schema = factory.newSchema(XsdReceiver.receive(this.schemaLocation));
        Validator validator = schema.newValidator();

        validator.setResourceResolver(factory.getResourceResolver());
        validator.validate(new StreamSource(new ByteArrayInputStream(data)));
    } catch (Exception e) {
        e.printStackTrace();
        return false;
    }

    return true;
}
public class XsdReceiver implements LSResourceResolver {
    public static Source receive(String url) {
        url = url.contains(" ") ? url.replace(" ", "/") : url;
        HttpURLConnection c = XsdReceiver.followRedirection(url);

        if (c == null)
            return null;

        try {
            return new StreamSource(c.getInputStream());
        } catch (IOException e) {
            return null;
        }
    }

    private static HttpURLConnection followRedirection(String url) {
        HttpURLConnection c = null;

        try {
            URL u = new URL(url);
            c = (HttpURLConnection)u.openConnection();
            int status = c.getResponseCode();

            if (status == HttpURLConnection.HTTP_MOVED_TEMP
                    || status == HttpURLConnection.HTTP_MOVED_PERM
                    || status == HttpURLConnection.HTTP_SEE_OTHER
            )
                c = (HttpURLConnection) new URL(c.getHeaderField("Location")).openConnection();
        } catch (IOException e) {
            return null;
        }

        return c;
    }

    @Override
    public LSInput resolveResource(String type, String namespaceURI, String publicId, String systemId, String baseURI) {
        if (namespaceURI == null || type.equals("http://www.w3.org/TR/REC-xml"))
            return null;

        if (systemId.equals("xml.xsd")) {
            try {
                File f = ResourceUtils.getFile("classpath:data/xml.xsd");

                return new Input(StandardCharsets.UTF_8.name(), new FileInputStream(f), null, systemId, publicId, namespaceURI, true, null);
            } catch (IOException e) {
                return null;
            }
        }

        String file = String.join("/", namespaceURI, systemId);
        HttpURLConnection connection;
        Input i = null;

        connection = XsdReceiver.followRedirection(file);

        if (connection == null)
            return null;

        try {
            i = new Input(StandardCharsets.UTF_8.name(), connection.getInputStream(), null, systemId, publicId, namespaceURI, true, null);
        } catch (IOException e) {
            return null;
        }

        return i;
    }
}
public class Input implements LSInput {
    private String encoding;
    private InputStream byteStream;
    private String stringData;
    private String systemId;
    private String publicId;
    private String baseUri;
    boolean certifiedText;
    private Reader characterStream;

    public Input(String encoding, InputStream byteStream, String stringData, String systemId, String publicId, String baseUri, boolean certifiedText, Reader characterStream) {
        this.encoding = encoding;
        this.byteStream = byteStream;
        this.stringData = stringData;
        this.systemId = systemId;
        this.publicId = publicId;
        this.baseUri = baseUri;
        this.certifiedText = certifiedText;
        this.characterStream = characterStream;
    }

    @Override
    public Reader getCharacterStream() {
        return this.characterStream;
    }

    @Override
    public void setCharacterStream(Reader characterStream) {
        this.characterStream = characterStream;
    }

    @Override
    public InputStream getByteStream() {
        return this.byteStream;
    }

    @Override
    public void setByteStream(InputStream byteStream) {
        this.byteStream = byteStream;
    }

    @Override
    public String getStringData() {
        return this.stringData;
    }

    @Override
    public void setStringData(String stringData) {
        this.stringData = stringData;
    }

    @Override
    public String getSystemId() {
        return this.systemId;
    }

    @Override
    public void setSystemId(String systemId) {
        this.systemId = systemId;
    }

    @Override
    public String getPublicId() {
        return this.publicId;
    }

    @Override
    public void setPublicId(String publicId) {
        this.publicId = publicId;
    }

    @Override
    public String getBaseURI() {
        return this.baseUri;
    }

    @Override
    public void setBaseURI(String baseURI) {
        this.baseUri = baseURI;
    }

    @Override
    public String getEncoding() {
        return this.encoding;
    }

    @Override
    public void setEncoding(String encoding) {
        this.encoding = encoding;
    }

    @Override
    public boolean getCertifiedText() {
        return this.certifiedText;
    }

    @Override
    public void setCertifiedText(boolean certifiedText) {
        this.certifiedText = certifiedText;
    }
}
Related