Comparing FISTA, ADMM and Proximal Newton

This tutorial shows how to compare perfomance of FISTA, ADMM and Proximal Newton. We will use ‘heart’ dataset for the demonstration.

library(solvers)
library(lattice)

Let’s load the data..

x <- SLOPE::heart$x
y <- SLOPE::heart$y

First step is generating fits for each of the algorithms. Note that diagnostics=TRUE flag is necessary so that solver records the metrics at each iteration.

# Obtaining the fit for the solvers we want to compare
fista_fit <- FISTA(x, y, family="binomial", alpha=(0.01),diagnostics=TRUE)
admm_fit <- ADMM(x, y, family="binomial", alpha=(0.01),diagnostics=TRUE)
pn_fit <- PN(x, y, family="binomial", alpha=(0.01),diagnostics=TRUE)

We haven’t specified optimization algorithm choice in ADMM so default (L-BFGS) will be used.

To compare total execution time, we can use total_time attribute of the fit.

# Comparing total execution time
print(fista_fit$total_time)
#> [1] 0.1129562
print(admm_fit$total_time)
#> [1] 0.4189371
print(pn_fit$total_time)
#> [1] 6.784939

Now, these fits cannot be directly used to plot as they contain a lot of parameter so we’ll call a utility function that merges the relevant parameter and returns a dataframe.

# Merging the metrics into one dataframe
f <- mergeFits(list(fista_fit, admm_fit, pn_fit), cutoff_time=1.5)

Finally, now we can generate the loss vs time plot.

# Plotting objective vs time
plt <- xyplot(loss ~ time, group = solver, data = f, auto.key = list(corner = c(0.9, 0.9)), type = "l")
update(plt, par.settings = list(fontsize = list(text = 18)))

Akarsh Goyal

2020-08-31