This phase I trial studies the side effects and the best dose of nivolumab when given together with gene-modified T cells and vaccine therapy in treating patients with solid tumors that express the cancer-testes antigen NY-ESO-1 gene AND have spread from where it started to nearby tissue or lymph nodes (locally advanced) or distant organs (stage IV). T cells are a special type of white blood cells (immune cell) that have the ability to kill cancer cells. Nivolumab may block PD-1 which is found on T cells and help the immune system kill cancer cells. Placing a modified gene for the NY-ESO-1 T cell receptor (TCR) into the patients' T cells in the laboratory and then giving them back to the patient may help the body build an immune response to kill tumor cells that express NY-ESO-1. Dendritic cells are another type of blood cell that can teach other cells in the body to look for cancer cells and attack them. Giving a dendritic cell vaccine with the NY-ESO-1 protein may help dendritic cells teach the immune system to target cancer cells expressing that protein, and further help the T cells attack cancer. Giving nivolumab together with gene-modified T-cells and dendritic cell vaccine may teach the immune system to recognize and kill cancer cells that express NY-ESO-1.