Week - 1 - Daa - SM
Week - 1 - Daa - SM
Evolution Of algorithm
The first ever algorithm was proposed by Ada Lovelace
Mathematician who used the concepts of zero and decimal
position of number.
During the 1940’s and 1950’s research oriented towards building
efficient computer system , so that they can be used in
scientific, commercial engineering problems.
Structured Programming came into existence after Alan Turning
introduced the idea of effective procedure in 1936.
Donald Knuth is the father of algorithms.
What is an algorithm?
An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for
obtaining a required output for any legitimate input in a finite amount of time.
Problem
Algorithm
Input Computer
Output
Following are the features which an algorithm has:
1. Finiteness
terminates after a finite number of steps
2. Definiteness
rigorously and unambiguously specified
5. Effectiveness
steps are sufficiently simple and basic
Why study algorithms?
Theoretical importance
Practical importance
11
Choosing Pivot
• Always pick the first element as a pivot.
• Always pick the last element as a pivot
• Pick a random element as a pivot.
• Pick the middle as the pivot.
Consider: arr[] = {10, 80, 30, 90, 40}.
• Compare 10 with the pivot and as it is less than pivot arrange it accordingly .
Compare 80 with the pivot. It is greater than pivot.
As the partition process is done recursively, it keeps on putting the pivot in its actual position in the
sorted array. Repeatedly putting pivots in their actual position makes the array sorted
Step 1 − Choose the highest index value has pivot
Step 2 − Take two variables to point left and right of the list excluding pivot Step 3 − left points to the low index
Step 4 − right points to the high
Step 5 − while value at left is less than pivot move right
Step 6 − while value at right is greater than pivot move left
Step 7 − if both step 5 and step 6 does not match swap left and right
Step 8 − if left ≥ right, the point where they met is new pivot
Pseudocode
procedure quickSort(left, right)
if right-left <= 0
return
else
pivot = A[right]
partition = partitionFunc(left, right, pivot)
quickSort(left,partition-1)
quickSort(partition+1,right)
end if
end procedure
Complexity Analysis of Quick Sort:
Time Complexity:
Best Case: Ω (N log (N))
The best-case scenario for quicksort occur when the pivot chosen at the each step
divides the array into roughly equal halves.
In this case, the algorithm will make balanced partitions, leading to efficient
Sorting.
Average Case: θ ( N log (N))
Quicksort’s average-case performance is usually very good in practice, making it
one of the fastest sorting Algorithm.
Worst Case: O(N2)
The worst-case Scenario for Quicksort occur when the pivot at each step
consistently results in highly unbalanced partitions.
Auxiliary Space: O(1), if we don’t consider the recursive stack space. If we
consider the recursive stack space then, in the worst case quicksort could
make O(N).
Mergesort
Merge sort is a sorting technique based on divide and conquer technique. With worst-case time
complexity being Ο(n log n), it is one of the most used and approached algorithms.
The MergeSort function keeps on splitting an array into two halves until a condition is met where we
try to perform MergeSort on a subarray of size 1, i.e., p == r.
And then, it combines the individually sorted subarrays into larger arrays until the whole array is
merged.
ALGORITHM-MERGE SORT
1. If p<r
2. Then q → ( p+ r)/2
3. MERGE-SORT (A, p, q)
4. MERGE-SORT ( A, q+1,r)
5. MERGE ( A, p, q, r)
Here we called MergeSort(A, 0, length(A)-1) to sort the complete array.
Procedure
What is a recurrence relation?
A recurrence relation, T(n), is a recursive function of an integer variable n.
Like all recursive functions, it has one or more recursive cases and one or
more base cases.
Example:
The portion of the definition that does not contain T is called the base case of
the recurrence relation; the portion that contains T is called the recurrent or
recursive case.
Recurrence relations are useful for expressing the running times (i.e., the
number of basic operations executed) of recursive algorithms
.
Forming Recurrence Relations
For a given recursive method, the base case and the recursive case of its recurrence relation
correspond directly to the base case and the recursive case of the method.
Example 1: Write the recurrence relation for the following method:
public void f (int n) {
if (n > 0) {
System.out.println(n);
f(n-1);
}
}
The base case is reached when n = = 0. The method performs one comparison. Thus, the number of
operations when n = = 0, T(0), is some constant a.
When n > 0, the method performs two basic operations and then calls itself, using ONE recursive
call, with a parameter n – 1.
Therefore the recurrence relation is:
T(0) = a for some constant a
T(n) = b + T(n – 1) for some constant b
• In General, T(n) is usually a sum of various choices of T(m ), the cost of the recursive
sub problems, plus the cost of the work done outside the recursive calls:
T(n ) = aT(f(n)) + bT(g(n)) + . . . + c(n)
where a and b are the number of subproblems, f(n) and g(n) are subproblem sizes,
and
c(n) is the cost of the work done outside the recursive calls [Note: c(n) may be a
constant]
Example
Example 1: Write the recurrence relation for the following method:
The base case is reached when n == 1. The method performs one comparison and one
return statement. Therefore, T(1), is some constant c.
When n > 1, the method performs TWO recursive calls, each with the parameter n /
2, and some constant # of basic operations.
For recurrence relation T(n) = 2T(n/2) + cn, the values of a = 2, b = 2 and k =1. Here logb(a) =
log2(2) = 1 = k. Therefore, the complexity will be Θ(nlog2(n)). Similarly for recurrence
relation T(n) = 2T(n/2) + √n, the values of a = 2, b = 2 and k =1/2. Here logb(a) = log2(2) = 1 >
k. Therefore, the complexity will be Θ(n).
Type 2: Linear Recurrence Relations:
Following are some of the examples of recurrence relations based on linear
recurrence relation.
T(n) = T(n-1) + n for n>0 and T(0) = 1
Substituting k = n, we get
T(n) = T(0) + 1 + 2+….. +n = n(n+1)/2 = O(n^2)
Type 3: Value Substitution Before Solving:
Sometimes, recurrence relations can’t be directly solved using techniques like
substitution, recurrence tree or master method. Therefore, we need to
convert the recurrence relation into appropriate form before solving. For
example,
T(n) = T(√n) + 1
The complexity of telescoping is a topic in mathematics that deals with the computational
complexity of the creative telescoping method for definite integration of special functions.
The goal is to obtain fast algorithms and implementations for definite integration of general
special functions, in a complexity-driven perspective.
Master Theorem (Master Method)
The master method provides an estimate of the growth rate of the solution for recurrences of the
form:
It is used for calculation of time complexity. Only specific recurrence relations can be solved using
master theorem.
If T(n) is interpreted as the number of steps needed to execute an algorithm for an input of size n,
this recurrence corresponds to a “divide and conquer” algorithm, in which a problem of size n is
divided into a sub-problems of size n / b, where a, b are positive constants:
Example1: Find the big-Oh running time of the following recurrence. Use the Master
Theorem:
T(1) = 1
T(n) = 2T(n / 2) + n
Solution: a = 2, b = 2, c = 1 a = bc Case 2
Hence T(n) is O(n log n)
Example2: Find the big-Oh running time of the following recurrence. Use the Master
Theorem:
T(1) = 1
T(n) = 4T(n / 2) + kn3 + h where k ≥ 1 and h 1
Solution: a = 4, b = 2, c = 3 a < bc Case 3
Hence T(n) is O(n3)