Unsuccessful attempts to obtain improvement in ANN regression model results (KERAS)

by Corse   Last Updated February 11, 2019 09:26 AM - source

I am attempting to build a ANN regression model in KERAS with 3 inputs and 3 outputs (I have attached link to the data I was trying on). Also, I am using Hyperas to optimize the hyperparameters.

My pseudocode is as follows:


def create_model()

4-layers using 4 instances of:    
Dense({{choice([np.power(2,1),np.power(2,2),np.power(2,3)])}}, activation={{choice(['tanh','relu', 'sigmoid'])}})(inputs1)
Dropout({{uniform(0, 1)}})

3-outputs are defined using 3 instances of:
Dense(1,  activation={{choice(['tanh','relu', 'sigmoid'])}})

activation_fn (adam, SGD, rmsprop) are given by:
activation_fn = keras.optimizers.activation_fn (lr={{choice([10**-3,10**-2, 10**-1])}})

model.fit(inputs, outputs,

I ran the current formulation for the above problem, with number of evaluation runs set at 100. Using either 'mae' or 'mse' as the error, I am getting an actual mean error of 225, 0.25, 45 for each of the outputs respectively.

I have also attempted setting a high number of neurons such as np.power(2,10), however it appears that the error remains stuck at approximately the values above.

Anyone knows if there is any way at all where I can obtain lower errors on the network? I have also tried weighting the losses from each output, and it didn't seem to help. Priority for error reduction is on output 1, then output 3 then output 2.

Appreciate your advice on this, have been hitting the wall on this one. Thank you

Tags : keras

Related Questions

Inverting Model.Add() layer in Keras

Updated January 14, 2018 08:26 AM

tf.keras: Feeding extra values to model.fit

Updated April 15, 2019 09:26 AM