Inclusion Criteria:

          -  INCLUSION CRITERIA FOR STRATA A, B, D AND E

          -  Tumor: patient must have one of the following diagnoses to be eligible:

          -  Stratum A, currently closed to enrollment: Patients must have a recurrent, progressive
             or refractory DIPG following radiation therapy with or without chemotherapy

               -  Histologic diagnosis is not required for patients with typical imaging findings
                  of DIPG (defined as patients with a diffuse expansile mass centered in and
                  involving at least 2/3 of the pons); patients with brainstem tumors who have
                  undergone biopsy with a diagnosis of high-grade glioma or diffuse infiltrating
                  glioma are also eligible

          -  Stratum B: Patients must have a histologically confirmed diagnosis of a non-brainstem
             high-grade glioma (NB-HGG) that is recurrent, progressive or refractory following
             therapy which included radiotherapy; spinal primary disease is eligible

          -  Stratum D: Patients must have a histologically confirmed diagnosis of ependymoma that
             is recurrent, progressive or refractory following therapy which included radiotherapy

          -  Stratum E: Patients must have a histologically confirmed diagnosis of medulloblastoma
             that is recurrent, progressive or refractory following therapy which included
             radiotherapy

          -  Patients must have adequate pre-trial formalin-fixed paraffin-embedded (FFPE) tumor
             material available for use in the biology studies mutational analysis and genome wide
             sequencing for each stratum

               -  Patients with DIPG who have tissue available are requested to submit similar
                  tissue as patients in other strata; however, this is not required for eligibility

          -  All subjects must have measurable disease in 2-dimensions on MRI scan of the brain;
             disease should be consistently measured with the two largest perpendicular dimensions

          -  Patients must have received prior radiation therapy and/or chemotherapy and recovered
             from the acute treatment related toxicities (defined as =< grade 1 if not defined in
             eligibility criteria) of all prior chemotherapy, immunotherapy or radiotherapy prior
             to entering this study; there is no upper limit to the number of prior therapies that
             is allowed

          -  Patients must have received their last dose of known myelosuppressive anticancer
             therapy at least three (3) weeks prior to study enrollment or at least six (6) weeks
             if prior nitrosourea

          -  Biologic or investigational agent (anti-neoplastic): Patient must have received their
             last dose of the investigational or biologic agent >= 7 days prior to study enrollment

               -  For agents that have known adverse events occurring beyond 7 days after
                  administration, this period must be extended beyond the time during which adverse
                  events are known to occur; the duration must be discussed with and approved by
                  the study chair

          -  Monoclonal antibody treatment and/or agents with prolonged half-lives: Patient must
             have recovered from any acute toxicity potentially related to the agent and received
             their last dose of the agent >= 28 days prior to study enrollment

          -  Patient must have completed immunotherapy (e.g. tumor vaccines, oncolytic viruses,
             etc.) at least 42 days prior to enrollment

          -  Patients must have had their last fraction of:

               -  Craniospinal irradiation >= 3 months prior to enrollment

               -  Other substantial bone marrow irradiation >= 6 weeks prior to enrollment

               -  Local palliative radiation therapy (XRT) (small port) >= 2 weeks

          -  Patient must be >= 12 weeks since autologous bone marrow/stem cell transplant prior to
             enrollment

          -  Patients must be fully recovered from all acute effects of prior surgical intervention

          -  All races and ethnic groups are eligible for this study

          -  Patients with neurological deficits should have deficits that are completely stable
             for a minimum of 1 week (7 days) prior to enrollment

          -  Karnofsky performance scale (KPS for > 16 years of age) or Lansky performance score
             (LPS for =< 16 years of age) assessed within two weeks of enrollment must be >= 70;
             patients who are unable to walk because of neurologic deficits, but who are up in a
             wheelchair, will be considered ambulatory for the purpose of assessing the performance
             score

          -  Absolute neutrophil count >= 1000 cells/uL

          -  Platelets >= 75,000 cells /uL (unsupported, defined as no platelet transfusion within
             7 days)

          -  Hemoglobin >= 8 g/dl (may receive transfusions)

          -  Total bilirubin =< 1.5 times institutional upper limit of normal (ULN)

          -  Alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase [SGPT]) =< 3 x
             institutional upper limit of normal

          -  Albumin >= 2 g/dl

          -  Serum creatinine based on age/gender as noted below; patients that do not meet the
             criteria below but have a 24 hour creatinine clearance or glomerular filtration rate
             (GFR) (radioisotope or iothalamate) >= 70 ml/min/1.73 m^2 are eligible

               -  Age: 1 to < 2 years; maximum serum creatinine (mg/dL): 0.6 (male); 0.6 (female)

               -  Age: 2 to < 6 years; maximum serum creatinine (mg/dL): 0.8 (male); 0.8 (female)

               -  Age: 6 to < 10 years; maximum serum creatinine (mg/dL): 1 (male); 1 (female)

               -  Age: 10 to < 13 years; maximum serum creatinine (mg/dL): 1.2 (male); 1.2 (female)

               -  Age: 13 to < 16 years; maximum serum creatinine (mg/dL): 1.5 (male); 1.4 (female)

               -  Age: >= 16 years; maximum serum creatinine (mg/dL): 1.7 (male); 1.4 (female)

          -  Pulse oximetry > 93% on room air and no evidence of dyspnea at rest

          -  Human immunodeficiency virus (HIV)- infected patients on effective anti-retroviral
             therapy with undetectable viral load within 6 months are eligible for this trial

          -  Patients must be off all colony-forming growth factor(s) for at least 1 week prior to
             registration (e.g. filgrastim, sargramostim, erythropoietin); 2 weeks must have
             elapsed for long-acting formulations

          -  Patients must be willing to use brief courses (at least 72 hours) of steroids as
             directed for potential inflammatory side effects of the therapy if recommended by
             their treating physician

          -  Female subjects of childbearing potential must not be pregnant or breast-feeding;
             female patients of childbearing potential must have a negative serum or urine
             pregnancy test within 72 hours prior to receiving the first dose of study medication;
             if the urine test is positive or cannot be confirmed as negative, a serum pregnancy
             test will be required; pregnant women are excluded from this study because
             pembrolizumab (MK-3475) is an agent with the potential for teratogenic effects;
             because there is unknown but potential risk for adverse events in nursing infants
             secondary to treatment of the mother with pembrolizumab (MK-3475), breastfeeding
             should be discontinued if the mother is to be treated with pembrolizumab (MK-3475)

          -  Patients of childbearing or child fathering potential must be willing to use 2 methods
             of birth control or be surgically sterile or abstain from heterosexual activity while
             being treated on this study and for 6 months after the last dose of study medication

          -  The patient or parent/guardian is able to understand the consent and is willing to
             sign a written informed consent document, inclusive of assent where appropriate,
             according to institutional guidelines

          -  INCLUSION CRITERIA FOR STRATUM C: Diagnosis of hypermutated brain tumors Patients with
             brain tumors and increased tumor mutation burden as determined by

               -  Confirmed diagnosis of CMMRD syndrome by Clinical Laboratory Improvement Act
                  (CLIA)-certified germline gene sequencing OR

               -  Confirmation of high mutation burden by whole genome/exome sequencing performed
                  in a CLIA-certified laboratory and/or the use of Foundation One next generation
                  sequence panel or another CLIA approved targeted sequencing lab with publicly
                  available correlations between number of mutations found in the panel and
                  mutations per megabase and/or genome; for protocol purposes a high mutation
                  burden will be defined as at least 180 non-synonymous coding-region mutations by
                  whole exome/genome sequencing (well above two standard deviations of the number
                  of median similar mutations described in pediatric CNS cancers) AND/OR a high
                  tumor mutation burden (TMB) or intermediate TMB based on the reporting parameters
                  of the panel; TMB parameters provided for the Foundation One reports are high
                  tumor mutation burden is >= 20 mutations per megabase or intermediate TMB is
                  between 6 to 19 mutations per megabase OR

               -  Confirmed diagnosis of Lynch syndrome by CLIA-certified germline gene sequencing;
                  patients with Lynch syndrome will not be accounted for in primary objective
                  unless their tumors are determined to have the minimum number of mutations
                  described above but they will still be eligible for this study

                    -  Low-grade tumors in patients with CMMRD or Lynch syndrome do not have to
                       reach the threshold of 100 mutations for study inclusion

          -  INCLUSION CRITERIA FOR STRATUM C: Patients must have a histologically confirmed
             primary brain tumor that is recurrent, progressive or refractory; inclusion criteria
             encompasses all types of brain tumors (e.g. gliomas, embryonal tumors or any other
             type of brain tumor as long as other eligibility criteria are met;

               -  Patients with high-grade gliomas are eligible for this clinical trial at least 2
                  weeks after completion of radiotherapy independent of tumor
                  progression/recurrence as long as they are not enrolled on any other therapeutic
                  clinical trial and there is macroscopic residual disease

               -  Patients with other concomitant tumors associated with CMMRD syndrome including
                  gastrointestinal polyps/adenomas and carcinomas, lymphomas and leukemias will be
                  eligible as long as they are not requiring anticancer therapy directed against
                  these other cancers and meet all other eligibility criteria

          -  INCLUSION CRITERIA FOR STRATUM C: Patients must have adequate pre-trial FFPE tumor
             material available and be willing to provide a blood sample for use in the genome wide
             sequencing studies; while tissue is required for genome-wide sequencing of tumor and
             germline samples, patients will be deemed eligible for the study with a minimum of
             approximately 10 unstained slides for the planned analysis

          -  INCLUSION CRITERIA FOR STRATUM C: Subjects must have measurable disease in
             2-dimensions on MRI scan of the brain and/or spine with the exception allowed for
             non-progressed HGGs; disease should be consistently measured with the two largest
             perpendicular dimensions

          -  INCLUSION CRITERIA FOR STRATUM C: Patients must have received prior radiotherapy
             and/or chemotherapy with the following exceptions:

               -  Patients with secondary CNS cancers after a previous medical problem/malignancy
                  who cannot receive full dose of radiotherapy (> 50 Gy) as long as they meet all
                  other eligibility criteria

               -  Patients with progressive low-grade gliomas and CMMRD or Lynch syndrome Patients
                  must have recovered from the acute treatment related toxicities (defined as =<
                  grade 1 if not defined in eligibility criteria) of all prior chemotherapy,
                  immunotherapy or radiotherapy prior to entering this study; there is no upper
                  limit to the number of prior therapies that is allowed

          -  INCLUSION CRITERIA FOR STRATUM C: Patients must have received their last dose of known
             myelosuppressive anticancer therapy at least three (3) weeks prior to study enrollment
             or at least six (6) weeks if prior nitrosourea

          -  INCLUSION CRITERIA FOR STRATUM C: Patient must have received their last dose of the
             investigational or biologic agent >= 7 days prior to study enrollment

               -  For agents that have known adverse events occurring beyond 7 days after
                  administration, this period must be extended beyond the time during which adverse
                  events are known to occur; the duration must be discussed with and approved by
                  the study chair

          -  INCLUSION CRITERIA FOR STRATUM C: Monoclonal antibody treatment and/or agents with
             prolonged half-lives: Patient must have recovered from any acute toxicity potentially
             related to the agent and received their last dose of the agent >= 28 days prior to
             study enrollment

          -  INCLUSION CRITERIA FOR STRATUM C: Patient must have completed immunotherapy (e.g.
             tumor vaccines, oncolytic viruses, etc.) at least 42 days prior to enrollment

          -  INCLUSION CRITERIA FOR STRATUM C: Patients must have had their last fraction of:

               -  Craniospinal irradiation >= 3 months prior to enrollment

               -  Other substantial bone marrow irradiation >= 6 weeks prior to enrollment

               -  Local palliative radiation therapy (XRT) (small port) >= 2 weeks

          -  INCLUSION CRITERIA FOR STRATUM C: Patient must be:

               -  >= 12 weeks since autologous bone marrow/stem cell transplant prior to enrollment

               -  >= 5 years since allogeneic bone marrow transplant prior to enrollment with no
                  evidence of active graft versus (vs.) host disease

          -  INCLUSION CRITERIA FOR STRATUM C: Patients must be fully recovered from all acute
             effects of prior surgical intervention

          -  INCLUSION CRITERIA FOR STRATUM C: All races and ethnic groups are eligible for this
             study

          -  INCLUSION CRITERIA FOR STRATUM C: Patients with neurological deficits should have
             deficits that are completely stable for a minimum of 1 week (7 days) prior to
             enrollment

          -  INCLUSION CRITERIA FOR STRATUM C: Karnofsky performance scale (KPS for > 16 years of
             age) or Lansky performance score (LPS for =< 16 years of age) assessed within two
             weeks of enrollment must be >= 60; patients who are unable to walk because of
             neurologic deficits, but who are up in a wheelchair, will be considered ambulatory for
             the purpose of assessing the performance score

          -  INCLUSION CRITERIA FOR STRATUM C: Absolute neutrophil count >= 1000 cells/uL

          -  INCLUSION CRITERIA FOR STRATUM C: Platelets >= 75,000 cells/uL (unsupported, defined
             as no platelet transfusion within 7 days)

          -  INCLUSION CRITERIA FOR STRATUM C: Hemoglobin >= 8 g/dl (may receive transfusions)

          -  INCLUSION CRITERIA FOR STRATUM C: Total bilirubin =< 1.5 times institutional upper
             limit of normal (ULN)

          -  INCLUSION CRITERIA FOR STRATUM C: ALT (SGPT) =< 3 x institutional upper limit of
             normal

          -  INCLUSION CRITERIA FOR STRATUM C: Albumin >= 2 g/dl

          -  INCLUSION CRITERIA FOR STRATUM C: Serum creatinine based on age/gender as noted below;
             patients that do not meet the criteria below but have a 24 hour creatinine clearance
             or GFR (radioisotope or iothalamate) >= 70 ml/min/1.73 m^2 are eligible

               -  Age: 1 to < 2 years; maximum serum creatinine (mg/dL): 0.6 (male); 0.6 (female)

               -  Age: 2 to < 6 years; maximum serum creatinine (mg/dL): 0.8 (male); 0.8 (female)

               -  Age: 6 to < 10 years; maximum serum creatinine (mg/dL): 1 (male); 1 (female)

               -  Age: 10 to < 13 years; maximum serum creatinine (mg/dL): 1.2 (male); 1.2 (female)

               -  Age: 13 to < 16 years; maximum serum creatinine (mg/dL): 1.5 (male); 1.4 (female)

               -  Age: >= 16 years; maximum serum creatinine (mg/dL): 1.7 (male); 1.4 (female)

          -  INCLUSION CRITERIA FOR STRATUM C: Pulse oximetry > 93% on room air and no evidence of
             dyspnea at rest

          -  INCLUSION CRITERIA FOR STRATUM C: HIV- infected patients on effective anti-retrovira