This study examines risk-return efficiency frontier and risk determinants for 36 banking systems of the European countries. The banks of European developed countries appeared more efficient than the banks of the transition and South East European (SEE) countries. In contrast to other studies, risk was measured as deviation from expected return that we derived through a utility maximization model. We found that volatility of return on assets (ROA) and return on equity (ROE) affects risk positively. In addition, we found that the banking systems of transition countries respond less to changes in volatility of ROA and ROE than the banking systems of European developed countries. Moreover, our robustness check model confirmed that the risk measure that was used can be explained by conventional risk proxies such us Z-score and equity ratio.