As the complexity of microsimulation models increases, concerns about model transparency are heightened.The authors conducted model "experiments" to explore the impact of variations in "deep" model parameters using 3 colorectal cancer (CRC) models. All natural history models were calibrated to match observed data on adenoma prevalence and cancer incidence but varied in their underlying specification of the adenocarcinoma process. The authors projected CRC incidence among individuals with an underlying adenoma or preclinical cancer v. those without any underlying condition and examined the impact of removing adenomas. They calculated the percentage of simulated CRC cases arising from adenomas that developed within 10 or 20 years prior to cancer diagnosis and estimated dwell time-defined as the time from the development of an adenoma to symptom-detected cancer in the absence of screening among individuals with a CRC diagnosis.The 20-year CRC incidence among 55-year-old individuals with an adenoma or preclinical cancer was 7 to 75 times greater than in the condition-free group. The removal of all adenomas among the subgroup with an underlying adenoma or cancer resulted in a reduction of 30% to 89% in cumulative incidence. Among CRCs diagnosed at age 65 years, the proportion arising from adenomas formed within 10 years ranged between 4% and 67%. The mean dwell time varied from 10.6 to 25.8 years.Models that all match observed data on adenoma prevalence and cancer incidence can produce quite different dwell times and very different answers with respect to the effectiveness of interventions. When conducting applied analyses to inform policy, using multiple models provides a sensitivity analysis on key (unobserved) "deep" model parameters and can provide guidance about specific areas in need of additional research and validation.