Decision making has evolved throughout the years, nowadays harnessing massive amounts and types of data through the unprecedented capabilities of data science, analytics, machine learning, and artificial intelligence. This has potentially led to higher quality and more informed decisions based on the collaborative rationality between humans and machines, no longer bounded by the cognitive capacity and limited rationality of each on their own. However, the multiplicity of modes of collaboration and interaction between humans and machines has also increased the complexity of decision making, consequentially complicating ex-ante and ex-post decision evaluation. Nevertheless, evaluation remains crucial to enable human and machine learning, rationalization, and sensemaking. This paper addresses the need for more research on why and how to evaluate collaborative rationality-based decisions, setting the stage for future studies in developing holistic evaluation solutions. By analyzing four relevant streams of literature: 1) classical decision theory and organizational management, 2) cognitive and neuroscience, 3) AI and ML, and 4) data-driven decision making, we highlight the limitations of current literature in considering a holistic evaluation perspective. Finally, we elaborate the theoretical underpinnings from the knowledge base on how humans and machines evaluate decisions, and the considerations for evaluating collaborative rationality-based decisions.