NotSerializableException: org.tartarus.snowball.ext.porterStemmer

[savicprvoslav]

Hello, I see that stemmer is not serializable :(

java.io.NotSerializableException: org.tartarus.snowball.ext.porterStemmer
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
    at weka.core.SerializationHelper.write(SerializationHelper.java:225)
    at weka.core.SerializationHelper.write(SerializationHelper.java:208)
    at 

[fracpete]

Did you try to serialize the snowball stemmer itself or the Weka stemmer wrapper? AFAIK, the Weka wrapper defines the snowball stemmer as transient and instantiates it on demand.

[savicprvoslav]

Hi,
I was serialising wrapper class that holds classifier instance with stringToVector filter that includes the weka.core.stemmers.SnowballStemmer. ( in its basics it is the same code as in your example)

Is this exception surprising to you, maybe I have done something wrong?

weka-dev 3.7.5

A bit off topic, I am including weka into alfresco as a automated document classification solution. So the stemmers are important to me.

[fracpete]

Hmm... Haven't actually used the stemmer in ages, merely turned them into a package. Will have a look when I'm back in the country next week.

[savicprvoslav]

Thanks, it would be awesome if I your code would help me in my quest.
I hope that you will solve the issue.

[fracpete]

Hmm... I just created a simple Maven project with a single class and I can't reproduce the non-serializable issue. The snowball stemmer artifact relies on Weka 3.7.12. Maybe your old Weka version is causing the conflict?

The test class looks as follows (in directory src/main/java/blah):

package blah;

import weka.core.Instances;
import weka.core.SerializationHelper;
import weka.core.converters.ConverterUtils;
import weka.core.stemmers.SnowballStemmer;
import weka.filters.Filter;
import weka.filters.unsupervised.attribute.StringToWordVector;
import java.io.File;

public class Blah {

  /**
   * for executing: mvn compile exec:java -Dfile=/some/where/data.arff 
   */
  public static void main(String[] args) throws Exception {
    String datafile = System.getProperty("file");
    System.out.println("Loading data: " + datafile);
    Instances data = ConverterUtils.DataSource.read(datafile);
    SnowballStemmer stemmer = new SnowballStemmer();
    stemmer.setStemmer("porter");
    StringToWordVector filter = new StringToWordVector();
    filter.setStemmer(stemmer);
    filter.setInputFormat(data);
    Instances filtered = Filter.useFilter(data, filter);
    String modelfile = System.getProperty("java.io.tmpdir") + File.separator + "s2vw.model";
    System.out.println("Saving filter model: " + modelfile);
    SerializationHelper.write(modelfile, filter);
    System.out.println("Loading filter model: " + modelfile);
    StringToWordVector filter2 = (StringToWordVector) SerializationHelper.read(modelfile);
    System.out.println(filter2);
  }
}

The associated pom.xml file is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

  <modelVersion>4.0.0</modelVersion>

  <groupId>nz.ac.waikato.cms.adams</groupId>
  <artifactId>snowball-test</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>snowball-test</name>
  <licenses>
    <license>
      <name>GNU General Public License 3</name>
      <url>http://www.gnu.org/licenses/gpl-3.0.txt</url>
    </license>
  </licenses>

  <developers>
    <developer>
      <id>fracpete</id>
      <name>Peter Reutemann</name>
      <email>fracpete@waikato.ac.nz</email>
      <url>http://www.cms.waikato.ac.nz/~fracpete/</url>
      <organization>University of Waikato, CS department</organization>
      <organizationUrl>http://www.cs.waikato.ac.nz/</organizationUrl>
      <roles>
        <role>architect</role>
        <role>developer</role>
      </roles>
      <timezone>+12</timezone>
    </developer>
  </developers>

  <dependencies>
    <dependency>
      <groupId>com.github.fracpete</groupId>
      <artifactId>snowball-stemmers-weka-package</artifactId>
      <version>1.0.1</version>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <version>2.3.2</version>
      </plugin>

      <!-- for executing: mvn compile exec:java -Dfile=/some/where/data.arff -->
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>1.2.1</version>
        <executions>
          <execution>
            <goals>
              <goal>java</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <mainClass>blah.Blah</mainClass>
          <workingDirectory>.</workingDirectory>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

[savicprvoslav]

Hi,
thank you for checking this out.

Yes mY current weka version is order than this in the example. Perhaps this is the issue, I will check it out and let you know about the results.

Thank you again.