Algorithms

Foundations: correctness and asymptotic analysis

10 / 454
Contents

Contents

Algorithms

0 of 454 complete

9 min read

Reading the constraints backwards

The largest input a problem allows is a message about which technique is expected. Learn to read it before you write anything.

Every problem worth solving comes with limits: the list has at most this many entries, the values go up to that. Those numbers look like a detail about testing. They are usually the strongest hint available about what kind of solution is wanted, and reading them first saves writing the wrong thing.

The reasoning runs backwards from a budget. A current machine executes very roughly 10810^8 to 10910^9 simple operations per second, so a program with a one second budget can afford about 10810^8 operations. Work out what each growth class evaluates to at the given size, and cross off everything above the budget.

What fits in a hundred million operationsPython

Change BUDGET to 10 ** 9 and run it again. Only one of the twelve verdicts changes.

At n=105n = 10^5, a quadratic solution needs 101010^{10} operations, which is around a hundred seconds. An nlog⁡nn \log n solution needs about 1.7×1061.7 \times 10^6, which is instant. There is no tuning that closes that gap, and knowing it before you start is the difference between an hour of profiling and a different algorithm.

The table to keep in your head

Largest n allowedIntended costWhat that usually means
n up to about 11Theta(n!)Try every ordering of the items
n up to about 22Theta(2^n) or Theta(2^n * n)Try every subset of the items
n up to about 100Theta(n^4)Four nested loops, or a table indexed by two pairs
n up to about 500Theta(n^3)Three nested loops, or a table filled in over pairs
n up to about 5000Theta(n^2)Every pair, or a table with one row per item
n up to about 10^6Theta(n log n)Sorting, or a pass with a logarithmic operation inside
n up to about 10^7Theta(n)A constant number of passes over the input
n up to 10^12 and beyondTheta(log n) or Theta(1)Arithmetic on the number itself, never a loop over it
Rough guidance for a budget of about a hundred million operations. The right-hand column is what a solution of that shape is usually built from.

The rows are not laws. They are what the person who set the limit was thinking. If the stated limit is n≤2×105n \le 2 \times 10^5, they have deliberately excluded n2n^2, which would be 4×10104 \times 10^{10}, and left room for nlog⁡nn \log n, which is about 3.5×1063.5 \times 10^6. If they had wanted a quadratic solution they would have written n≤3000n \le 3000.

The other numbers in the constraints

The size of the input is not the only limit worth reading. Several others change what is possible.

  • The range of the values. If the entries are at most 10610^6, an array with one slot per possible value is affordable and turns many searches into a direct lookup. If they are at most 101810^{18}, it is not.
  • The memory limit. A table of nn by nn numbers at n=105n = 10^5 is 101010^{10} entries. At 8 bytes each that is 80 gigabytes, so the algorithm is impossible for reasons that have nothing to do with time.
  • How many test cases there are. A limit of n≤105n \le 10^5 with up to 10410^4 separate cases is not a promise of 10510^5; look for a stated bound on the total, and if the total is what is bounded, budget against that.
  • Whether the values can be negative, zero, or repeated. These rarely change the growth and very often change whether your procedure is correct.

Here is a worked reading. A problem says: up to 2×1052 \times 10^5 items, each with a weight up to 10910^9, and you must report something about every pair. The pair count alone is about 2×10102 \times 10^{10}, so the answer cannot be produced by examining pairs one at a time. The weights being up to 10910^9 rules out an array indexed by weight, and also warns that a sum of weights can reach 2×10142 \times 10^{14}, which overflows a 32-bit integer and needs a 64-bit one. Both conclusions came from the constraints, before any algorithm existed.

A problem allows n up to 18 and asks for the best possible arrangement of n items. What should you try first, and why is n = 18 the giveaway?

Something that looks at every subset, costing about 2 to the 18, which is 262144, or 2 to the n times n, about 4.7 million. Both are comfortable. The giveaway is that 18 is far too small for any polynomial cost to be the reason for the limit: if a quadratic solution existed, the setter would have allowed n in the thousands. A limit near 20 is the standard signal that exponential work over subsets is expected. If it had said n up to 11, the signal would be every ordering instead, since 11 factorial is about 40 million and 12 factorial is about 479 million.

Check yourself

A problem states n up to 10^5 and a time limit of 1 second. You have a correct Theta(n^2) solution. What is the sensible next move?

All lessons in Algorithms