Algorithms & Programming

Lesson 3 of 6

Sorting and Recursion

Recursion and divide and conquer with the master theorem, the main sorting algorithms and their costs, counting inversions with merge sort, and binary search from sorted arrays to implied volatility.

Recursion and divide and conquer

A recursive function solves a problem by calling itself on a smaller version of it. It needs two parts: a base case that answers directly (an empty list is sorted) and a recursive step that shrinks the input and combines the results. Divide and conquer is the most common shape: split the input into pieces, solve each piece recursively, then combine.

The running time satisfies a recurrence. If you split into aa subproblems of size n/bn/b and spend O(nd)O(n^d) splitting and combining, then T(n)=a T(n/b)+O(nd)T(n) = a\,T(n/b) + O(n^d), and the master theorem reads off the answer by comparing dd with log⁡ba\log_b a:

T(n)={O(nd)d>log⁡baO(ndlog⁡n)d=log⁡baO(nlog⁡ba)d<log⁡baT(n) = \begin{cases} O(n^d) & d > \log_b a \\ O(n^d \log n) & d = \log_b a \\ O(n^{\log_b a}) & d < \log_b a \end{cases}

Three recurrences come up again and again. Merge sort is 2T(n/2)+O(n)2T(n/2) + O(n): here d=1=log⁡22d = 1 = \log_2 2, so O(nlog⁡n)O(n \log n). Binary search is T(n/2)+O(1)T(n/2) + O(1): d=0=log⁡21d = 0 = \log_2 1, so O(log⁡n)O(\log n). A routine that makes one linear pass and then recurses into only one half is T(n/2)+O(n)T(n/2) + O(n): d=1>0d = 1 > 0, so O(n)O(n), because the work shrinks geometrically (n+n/2+n/4+⋯≤2nn + n/2 + n/4 + \dots \le 2n).

One practical limit in Python: the default recursion depth is 1000. Halving recursion on a million elements goes about 20 levels deep and is fine. Recursion that peels off one element at a time on 10510^5 elements raises RecursionError, so write that version as a loop.