Inclusion Criteria:

          -  All patients and/or their parents or legal guardians must have the ability to
             understand and the willingness to sign a written informed consent

          -  Voluntary written informed consent must be obtained before performance of any
             study-related procedure not part of normal medical care, with the understanding that
             consent may be withdrawn by the subject at any time without prejudice to future
             medical care

          -  Weight over 40 kilograms (kg)

          -  Life expectancy of greater than 3 months

          -  Cohort #1: histologically confirmed CD20-positive, relapsed or refractory DLBCL,
             including de novo and transformed DLBCL (from follicular or marginal zone lymphoma);
             this includes patients with DLBCL who are found to have small cell infiltration of the
             bone marrow or other diagnostic material (representing a discordant lymphoma)

          -  Cohort #1: patients must be either refractory to or relapsed after up to 2 lines of
             prior therapy

          -  Cohort #2: histologic confirmation of relapsed or relapsed/refractory MCL confirmed by
             presence of cyclin D1 by immunohistochemistry (IHC) or fluorescence in situ
             hybridization (FISH)

          -  Patients must have measurable disease > 1.5 cm evidenced by computed tomography (CT)
             scan of the neck/chest/abdomen (abd)/pelvis or CT/positron emission tomography (PET)
             scans

          -  Cohort #2: patients must be either refractory to or relapsed after at least 1 line of
             prior therapy

          -  Eastern Cooperative Oncology Group (ECOG) performance score of 0-2

          -  Cohort #1: received only frontline CD20-directed immunotherapy with anthracycline- or
             anthracenedione-based multi-agent chemotherapy for patients with DLBCL; monotherapy
             rituximab or other CD20-directed immunotherapy prior to frontline chemotherapy, as
             maintenance therapy, and radiotherapy in a limited field or as a part of the frontline
             treatment plan are permitted; last treatment dose should be 3 weeks before start of
             study treatment

          -  Cohort #1: considered eligible for high-dose chemotherapy followed by autologous stem
             cell transplantation (ASCT)

          -  Cohort #2: patients with MCL with prior allogeneic hematopoietic stem cell transplant,
             minimum 6 months after transplant, not on immunosuppression, and without prior or
             active graft versus host disease (GVHD), are allowed

          -  Cohort #2: prior treatment with ibrutinib is allowed; patients should not have
             received any anti-lymphoma therapy within 3 weeks from start of study treatment, with
             the exception of ibrutinib

          -  Absolute neutrophil count (ANC) >= 1,000/mm^3 (performed within 14 days prior to day 1
             of protocol therapy)

               -  Filgrastim can be given prior to enrollment to achieve target ANC >= 1000/uL

          -  Platelets >= 50,000/mm^3 for MCL cohort and platelets >= 75,000/mm^3 for DLBCL cohort
             (performed within 14 days prior to day 1 of protocol therapy)

               -  NOTE: platelet transfusions and packed red blood cell transfusion can be given
                  prior to enrollment to achieve a target platelet (Plt) >= 50,000/uL for MCL and
                  >= 75,000/uL for DLBCL and hemoglobin of >= 8.5 g/dL

          -  Hemoglobin >= 8.5 g/dL (performed within 14 days prior to day 1 of protocol therapy)

          -  Total bilirubin within 1.5 x the upper limit of normal institutional limits; patients
             with elevation of unconjugated bilirubin alone, as in Gilbert's disease, are eligible
             (performed within 14 days prior to day 1 of protocol therapy)

          -  Aspartate aminotransferase (AST) =< 3 x upper limit of normal (ULN) unless
             demonstrated lymphoma involvement of the liver (performed within 14 days prior to day
             1 of protocol therapy)

          -  Alanine aminotransferase (ALT) =< 2.5 x ULN unless demonstrated lymphoma involvement
             of the liver (performed within 14 days prior to day 1 of protocol therapy)

          -  Creatinine clearance >= 50 mL/min for Cohort #1 (DLBCL) and >= 30 mL/min for Cohort #2
             (MCL) per the Cockcroft-Gault formula (performed within 14 days prior to day 1 of
             protocol therapy)

          -  If not receiving anticoagulants: international normalized ratio (INR) OR prothrombin
             (PT) =< 1.5 x ULN (performed within 14 days prior to day 1 of protocol therapy)

               -  If on anticoagulant therapy: PT must be within therapeutic range of intended use
                  of anticoagulants

          -  If not receiving anticoagulants: activated partial thromboplastin time (aPTT) =< 1.5 x
             ULN (performed within 14 days prior to day 1 of protocol therapy)

               -  If on anticoagulant therapy: aPTT must be within therapeutic range of intended
                  use of anticoagulants

          -  Female of childbearing potential: negative urine or serum pregnancy test, performed
             within 14 days prior to day 1 of protocol therapy

               -  If the urine test is positive or cannot be confirmed as negative, a serum
                  pregnancy test will be required

          -  Cardiac function (12 lead- electrocardiogram [ECG] versus [vs] non-12 lead ECG),
             performed within 14 days prior to day 1 of protocol therapy

          -  Female subjects must be either post-menopausal, surgically sterilized, or willing to
             use an acceptable method of birth control (i.e. a hormonal contraceptive,
             intra-uterine device, diaphragm with spermicide, condom with spermicide, or
             abstinence) beginning prior to study entry, for the duration of the study, and for six
             months following last dose of avelumab/utomilumab; should a woman become pregnant or
             suspect that she is pregnant while participating on the trial, she should inform her
             treating physician immediately

          -  Male subjects must agree to use an acceptable method of contraception beginning prior
             to study entry, for the duration of the study, and for six months following last dose
             of avelumab/utomilumab

        Exclusion Criteria:

          -  Patients who are not hematopoietic stem cell transplant candidates are excluded for
             the DLBCL cohort (cohort #1)

          -  Vaccinated with live, attenuated vaccines within 4 weeks of first dose of avelumab

          -  Patients may be on steroids prior to initiation of treatment, provided that, by cycle
             1 day 1, steroid use is tapered down to less than or equal to 10 mg/day of prednisone

          -  For cohort 1 (DLBCL) only: prior organ transplantation including allogeneic stem-cell
             transplantation

          -  For cohort 1 (DLBCL) only: prior RICE chemotherapy

          -  Patients with prior treatment with PD-1 or PD-L1 inhibitor

          -  Patients may not be receiving any other investigational agents, or concurrent
             biological therapy, chemotherapy, or radiation therapy

          -  Current use of immunosuppressive medication, except for the following: a. intranasal,
             inhaled, topical steroids, or local steroid injection (e.g., intra-articular
             injection); b. systemic corticosteroids at physiologic doses =< 10 mg/day of
             prednisone or equivalent; c. steroids as premedication for hypersensitivity reactions
             (e.g., CT scan premedication)

          -  For cohort 2 (MCL) only: strong CYP3A4 inducers/inhibitors within 14 days prior to day
             1 of protocol therapy and/or requires treatment with a strong cytochrome P450 (CYP)
             3A4/5 inhibitor

          -  CRITERIA SPECIFIC FOR COHORT #2 (MCL): Significant screening electrocardiogram (ECG)
             abnormalities including, but not limited to, left bundle branch block, 2nd degree
             atrioventricular (AV) block type II, 3rd degree block, or corrected QT interval (QTc)
             >= 470 msec; subjects with a cardiac pacemaker who have a QTc interval of >= 470 msec
             may be eligible if these findings are considered not clinically significant as
             documented via a cardiology evaluation

          -  CRITERIA SPECIFIC FOR COHORT #2 (MCL): Currently active, clinically significant
             hepatic impairment Child-Pugh class B or C according to the Child Pugh classification

          -  CRITERIA SPECIFIC FOR COHORT #2 (MCL): Unable to swallow capsules or malabsorption
             syndrome, disease significantly affecting gastrointestinal function, or resection of
             the stomach or small bowel, symptomatic inflammatory bowel disease or ulcerative
             colitis, or partial or complete bowel obstruction

          -  CRITERIA SPECIFIC FOR COHORT #2 (MCL): Known bleeding disorders (e.g., von
             Willebrand's disease) or hemophilia

          -  CRITERIA SPECIFIC FOR COHORT #2 (MCL): Major surgery within 4 weeks of first dose of
             study drug

          -  Active GVHD or on immunosuppressive medication for GVHD (applies to cohort #2)

          -  Recent infection requiring intravenous anti-infective treatment that was completed =<
             14 days before enrollment; active infection requiring systemic therapy

          -  Persisting toxicity related to prior therapy (National Cancer Institute [NCI] Common
             Terminology Criteria for Adverse Events [CTCAE] version [v]. 4.03 grade > 1); however,
             alopecia, sensory neuropathy grade =< 2, or other grade =< 2 not constituting a safety
             risk based on investigator's judgement are acceptable

          -  Hypersensitivity to chemotherapy or history of allergic reactions attributed to
             compounds of similar chemical or biologic composition to rituximab, ibrutinib, ICE,
             Ave, or Uto

          -  Patients should not have any uncontrolled illness including ongoing or active
             infection

          -  Any life-threatening illness, medical condition, or organ system dysfunction that, in
             the investigator's opinion, could compromise the subject's safety or put the study
             outcomes at undue risk

          -  Myocardial infarction within 6 months prior to enrollment or New York Heart
             Association (NYHA) class III or IV heart failure, uncontrolled angina, severe
             uncontrolled ventricular arrhythmias, or electrocardiographic evidence of acute
             ischemia or active conduction system abnormalities; prior to study entry, any ECG
             abnormality at screening has to be documented by the Investigator as not medically
             relevant

          -  Clinically significant (i.e., active) cardiovascular disease: cerebral vascular
             accident/stroke (< 6 months prior to enrollment), myocardial infarction (< 6 months
             prior to enrollment), unstable angina, congestive heart failure (>= New York Heart
             Association classification class II), or serious cardiac arrhythmia requiring
             medication

          -  Diagnosed or treated for another malignancy within 3 years of enrollment, with the
             exception of complete resection of basal cell carcinoma or squamous cell carcinoma of
             the skin, an in situ malignancy, or low-risk prostate cancer after curative therapy

          -  Patients with active central nervous system (CNS) disease or history of brain
             metastases are excluded from study

          -  Active human immunodeficiency virus (HIV) or hepatitis C virus (HCV) or hepatitis B
             virus (HBV); subjects who have an undetectable HIV viral load with CD4 >= 200 and are
             on highly active antiretroviral therapy (HAART) medication are allowed; subjects who
             are positive for hepatitis B core antibody or hepatitis B surface antigen must have a
             negative polymerase chain reaction (PCR) result before enrollment; those who are PCR
             positive will be excluded; patients who have had hepatitis C but have finished
             treatment and are PCR negative will be allowed; (testing to be done only in patients
             suspected of having infections or exposures)

          -  Active autoimmune disease that might deteriorate when receiving an immuno-stimulatory
             agent; patients with diabetes type I, vitiligo, psoriasis, or hypo- or hyperthyroid
             diseases not requiring immunosuppressive treatment are eligible

          -  Other severe acute or chronic medical conditions including immune colitis,
             inflammatory bowel disease, immune pneumonitis, pulmonary fibrosis or psychiatric
             conditions including recent (within the past year) or active suicidal ideation or
             behavior; or laboratory abnormalities that may increase the risk associated with study
             participation or study treatment administration or may interfere with the
             interpretation of study results and, in the judgment of the investigator, would make
             the patient inappropriate for entry into this study

          -  Pregnant women are excluded from this study; breastfeeding should be discontinued

          -  Other active malignancy

          -  Any other condition that would, in the investigator's judgement, contraindicate the
             patient's participation in the clinical study due to safety concerns with clinical
             study procedures

          -  Prospective participants who, in the opinion of the investigator, may not be able to
             comply with all study procedures (including compliance issues related to
             feasibility/logistics)

          -  Unable to understand the purpose and risks of the study and to provide a signed and
             dated informed consent form (ICF) and authorization to use protected health
             information (in accordance with national and local subject privacy regulations)