Inclusion Criteria:

          1. Neuroendocrine-like prostate cancer, based on histology OR based on clinical
             presentation as defined by meeting one of the two below criteria. All subjects must
             submit their primary tumor or metastatic biopsy pathology specimens to the Duke Cancer
             Institute where they will be centrally reviewed by Duke Pathology. Central Duke
             pathologic review is not required for screening but rather for confirmation of
             histologic subtype. Local pathologic review is sufficient for eligibility
             determination.

               1. Criterion 1: Presence of 1 of 3 histologically proven diagnoses: 1) Primary small
                  cell carcinoma of the prostate, defined by classic histologic features such as
                  small tumor cells with scanty cytoplasm, darkly stained nuclei with homogeneous
                  chromatin pattern. The tumor cells do not form glandular structure but grow as
                  solid sheets with frequent mitotic figures and necrosis; 2) Intermediate atypical
                  carcinoma of the prostate, which has histologic features distinct from small cell
                  carcinoma or adenocarcinoma. The tumor grows as solid sheets or vague glandular
                  structures. The tumor cells have moderate amounts of cytoplasm and centrally
                  located, round and regular nuclei with fine, granular and homogeneous chromatin.
                  Mitosis and necrosis are absent; 3) mixed histology tumors of the prostate,
                  containing both adenocarcinoma and neuroendocrine or small cell components.

               2. Criterion 2: Presence of histologically proven adenocarcinoma of the prostate
                  without any sign of neuroendocrine or small cell histology that is
                  radiographically progressing despite castrate levels of testosterone (<50 ng/mL)
                  with the following poor risk features:

             i. Prior progression despite therapy with either abiraterone acetate and/or
             enzalutamide.

             ii. At least one of the following: 1) Visceral metastases; 3) Bulky lymphadenopathy or
             pelvic mass (>5 cm); 4) Low PSA (<10 ng/mL) with high volume (>20) bone metastases; 5)
             Short interval (<6mo) to CRPC following initiation of hormonal therapy 6) Pathogenic
             alterations in two of the three following genes: TP53, RB1, and PTEN as determined by
             tissue or plasma tumor DNA commercial or academic assays. 7) Predominantly lytic bone
             metastases on imaging, 8) Presence of neuroendocrine markers on histology (positive
             staining of chromogranin A or synaptophysin) or in serum (abnormal high serum levels
             for chromogranin A or GRP) at initial diagnosis or at progression. Plus any of the
             following in the absence of other causes: A. elevated serum LDH (>= IULN); B.
             malignant hypercalcemia; C. elevated serum CEA (>2x IULN).

          2. Available archival tumor tissue for pathologic review and correlative studies. Tumor
             tissue (localized or metastatic) does not need to be received but rather identified
             and available (slides and/or blocks) to be sent to Duke.

          3. Documented progressive metastatic CRPC as determined by the provider based on at least
             one of the following criteria:

               1. PSA progression defined as 25% increase over baseline value with an increase in
                  the absolute value of at least 2.0 ng/mL that is confirmed by another PSA level
                  with a minimum of a 1 week interval and a minimum PSA of 2.0 ng/mL. Note: If
                  confirmed rise is the only indication of progression, a minimal starting value of
                  1.0 ng/mL is acceptable, unless pure small-cell carcinoma.

               2. Soft-tissue progression based on new lesions or growth of existing soft tissue
                  metastases.

               3. Progression of bone metastasis with one or more new bone lesion(s) by imaging.

          4. Castrate levels of serum total testosterone (<50 ng/dl) OR ongoing documented ADT
             unless pure small cell prostate cancer is present.

          5. Karnofsky performance status of 70 or higher.

          6. Acceptable initial laboratory values within 14 days of Cycle 1 Day 1

          7. Age >18

          8. Subjects with a partner who is a woman of child-bearing potential must agree to use
             one form of highly effective contraception as detailed in Section 8.3 of this protocol
             during the treatment period with cabazitaxel and for 3 months after the last dose of
             cabazitaxel and during the treatment period with nivolumab and for 7 months after the
             last dose of nivolumab, whichever is later. Subjects receiving cabazitaxel or
             nivolumab must also refrain from donating sperm during this period.

          9. Willing and able to provide written informed consent and HIPAA authorization for the
             release of personal health information.

         10. Life expectancy of over 3 months as determined by treating physician.

        Exclusion Criteria:

          1. Prior use of abiraterone or androgen receptor antagonists (ie enzalutamide) used to
             treat prostate cancer are permitted but should be stopped two or more weeks prior to
             study treatment initiation.

          2. Has received prior therapy with an anti-PD-1, anti-PD-L1, or anti PD L2 agent or with
             an agent directed to another stimulatory or co-inhibitory T-cell receptor (eg, CTLA-4,
             OX 40, CD137).

          3. Has received other prior systemic anti-cancer therapy including investigational agents
             within 4 weeks prior to study treatment initiation

          4. Prior receipt of cabazitaxel chemotherapy or 2 or more chemotherapy regimens. One
             prior chemotherapy regimen including docetaxel or platinum-containing chemotherapy is
             permitted.

          5. Has received prior radiotherapy within 2 weeks of start of study intervention.
             Participants must have recovered from all radiation-related toxicities, not require
             corticosteroids, and not have had radiation pneumonitis. A 1-week washout is permitted
             for palliative radiation (≤2 weeks of radiotherapy) to non-CNS disease.

          6. Has received a live vaccine within 30 days prior to the first dose of study drug.
             Examples of live vaccines include, but are not limited to, the following: measles,
             mumps, rubella, varicella/zoster (chicken pox), yellow fever, rabies, Bacillus
             Calmette-Guérin (BCG), and typhoid vaccine. Seasonal influenza vaccines for injection
             are generally killed virus vaccines and are allowed; however, intranasal influenza
             vaccines (eg, FluMist®) are live attenuated vaccines and are not allowed.

          7. Is currently participating in or has participated in a study of an investigational
             agent or has used an investigational device within 4 weeks prior to the first dose of
             study intervention.

          8. Has a diagnosis of immunodeficiency or is receiving chronic systemic steroid therapy
             (in dosing exceeding 10 mg daily of prednisone equivalent) or any other form of
             immunosuppressive therapy within 7 days prior to the first dose of study drug.

          9. Has a history of a second malignancy, unless potentially curative treatment has been
             completed with no evidence of malignancy for 2 years.

         10. Has known active untreated CNS metastases and/or carcinomatous meningitis.
             Participants with previously treated brain metastases may participate provided they
             are radiologically stable, i.e. without evidence of progression for at least 4 weeks
             by repeat imaging (note that the repeat imaging should be performed during study
             screening), clinically stable and without requirement of steroid treatment greater
             than prednisone 10mg (or equivalent) for at least 14 days prior to first dose of study
             intervention.

         11. Has active autoimmune disease that has required systemic treatment in the past 2 years
             (i.e. with use of disease modifying agents, corticosteroids or immunosuppressive
             drugs). Replacement therapy (eg., thyroxine, insulin, or physiologic corticosteroid
             replacement therapy for adrenal or pituitary insufficiency, etc.) is not considered a
             form of systemic treatment and is allowed.

         12. Has a history of (non-infectious) pneumonitis that required steroids or has current
             pneumonitis.

         13. Has an active infection requiring systemic therapy.

         14. Has a known uncontrolled Human Immunodeficiency Virus (HIV) infection based on
             detectable HIV viral load and abnormal CD4 count of <350/mm3.

         15. Has a known active Hepatitis B (defined as Hepatitis B surface antigen [HBsAg]
             reactive) or known active Hepatitis C virus (defined as HCV RNA [qualitative] is
             detected) infection.

         16. Has a known active TB (Bacillus Tuberculosis) infection.

         17. Has a history or current evidence of any condition, therapy, or laboratory abnormality
             that might confound the results of the study, interfere with the subject's
             participation for the full duration of the study, or is not in the best interest of
             the subject to participate, in the opinion of the treating investigator.

         18. Has known current psychiatric or substance abuse disorders that would interfere with
             cooperation with the requirements of the trial.

         19. Has had an allogenic tissue/solid organ transplant.