/home/andres/src/miniconda3/envs/speechActs/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/andres/src/miniconda3/envs/speechActs/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/andres/src/miniconda3/envs/speechActs/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/andres/src/miniconda3/envs/speechActs/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/andres/src/miniconda3/envs/speechActs/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/andres/src/miniconda3/envs/speechActs/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/home/andres/src/miniconda3/envs/speechActs/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/andres/src/miniconda3/envs/speechActs/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/andres/src/miniconda3/envs/speechActs/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/andres/src/miniconda3/envs/speechActs/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/andres/src/miniconda3/envs/speechActs/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/andres/src/miniconda3/envs/speechActs/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])

Required modules

Import data from file

import_data[source]

import_data(filepath)

Imports data from file and removes empty entries :param filepath: Path to file with SwDa utterances in format "ActTag,Text" :return: pandas dataframe with the data

Preprocess data, and divide it in train, validation and test sets

preprocess_data[source]

preprocess_data(data, lowercase=False, clean=False, remove_continuations=True)

Preprocess the data according to parameters. Divides the data into training, validation and test sets.

:param data: Pandas dataframe imported by import_data Optional: :param lowercase: Convert all text to lowercase :param clean: Remove punctuation marks and non-verbal utterances :param remove_continuations: Remove utterances with act tag "+"

:return: Pandas series with training, validation and test tags and utterances

Transform RandomBinaryProjections to transformer

class Transformer_RBP[source]

Transformer_RBP(hash_name='hasher', projection_count=1, rand_seed=None) :: BaseEstimator

Class that modifies RandomBinaryProjections to use as an sklearn transformer

Create input layer projections

This is the main contribution of the SGNN paper, and the implementation is inspired by Guillaume Chevalier's implementation, as well as his discussion with Sava Kalbachou.

build_input_layer[source]

build_input_layer(T=80, d=14)

Transformer to build the input layer, in SGNN style. Uses nltk skipgrams, and several Transformer_RBP layers as elements of the SGNN pipeline.

Create keras model

Following the SGNN paper's architecture

build_keras_model[source]

build_keras_model(train_labels)

Build keras model, with two hidden layers as the SGNN paper.

Main process, putting together previous functions

main[source]

main(lowercase=False, clean=False, remove_continuations=True)