I’m trying to struct a schema for db testing, and StructType apparently isn’t working for some reason. I’m following a tut, and it doesn’t import any extra module.

<type 'exceptions.NameError'>, NameError("name 'StructType' is not defined",), <traceback object at 0x2b555f0>)

I’m on spark 1.4.0, and Ubuntu 12 if that has anything to do with the problem. How would I fix this problem? Thank you in advance.

Did you import StructType? If not

from pyspark.sql.types import StructType

should solve the problem.

from pyspark.sql.types import StructType

That would fix it but next you might get NameError: name 'IntegerType' is not defined or NameError: name 'StringType' is not defined ..

To avoid all of that just do:

from pyspark.sql.types import *

Alternatively import all the types you require one by one:

from pyspark.sql.types import StructType, IntegerType, StringType

All Types: Apache Spark Data Types