Amid increasing calls for critical and anti-oppressive approaches to computer science (CS) education, educators are exploring how to create justice-centered teaching material. Additionally, broadening participation in justice-centered computing requires an understanding of students’ relationship with social justice and their CS education. In this study, we created and distributed a programming project with a social justice context and critical thinking reflection questions as a probe for an intermediate programming class. We conducted a thematic analysis of 11 semi-structured interviews and distributed a short survey (N=86) with students of this class at a large public research university in the American Midwest. Our findings showed that these students support social justice contexts and content within their computer science education. Students requested deeper dives and discussions into social justice programming that would challenge their preconceived notions, incorporate calls to action, and direct action. However, we also found an interesting tension forming: many students described how their homework problem-solving mindset clashed with the critical thinking reflection questions.