Udhay Prakash Blog

Saturday, March 22, 2025

Apache Airflow notes

Apache Airflow: A One-Stop Guide for Junior Developers

Apache Airflow is a powerful open-source tool for orchestrating jobs and managing data workflows. This guide covers everything you need—history, features, and a practical example—all explained simply.

History and Evolution

Workflow orchestration evolved over time. Here’s the journey:

Non/Quasi-Programmable Tools (e.g., Informatica, Talend):
Early tools like Informatica and Talend offered graphical interfaces for ETL workflows. While powerful for simple tasks, they weren’t fully programmable, limiting flexibility, dependency management, and version control.
cronTab and Event Scheduler:
Basic scheduling tools like cronTab (Linux) and Event Scheduler (Windows) ran jobs at fixed times but couldn’t handle dependencies or track job status.
Celery:
A step up, Celery provided a task queue with workers but required custom logic for workflows.
Apache Airflow (2014):
Created at Airbnb in 2014 and open-sourced in 2015, Airflow introduced code-defined workflows with dependency management, becoming an Apache project in 2016.

What is Airflow?

Airflow lets you programmatically define, schedule, and monitor workflows using Python. Workflows are represented as Directed Acyclic Graphs (DAGs)—tasks with a defined order and no loops.

Architecture with Mermaid Diagram

Here’s how Airflow’s components connect:

Web Server: Hosts the UI for monitoring.
Scheduler: Schedules tasks based on DAGs.
Executor: Manages task execution.
Workers: Run the tasks.
Metadata Database: Stores task states and logs.

Executor Modes with Mermaid Diagrams

LocalExecutor

Runs tasks on the same machine as the scheduler—simple but not scalable.

CeleryExecutor

Distributes tasks across workers using Celery—scalable but needs a broker like Redis.

Sample Python DAG Code

Here’s a styled Python DAG example that prints "Hello" and "World!":

from airflow import DAG
from airflow.operators.python import PythonOperator
from datetime import datetime

def print_hello():
    print("Hello")

def print_world():
    print("World!")

with DAG(
    dag_id='hello_world_dag',
    start_date=datetime(2023, 1, 1),
    schedule_interval='@daily',
    catchup=False
) as dag:
    task1 = PythonOperator(
        task_id='print_hello',
        python_callable=print_hello
    )
    task2 = PythonOperator(
        task_id='print_world',
        python_callable=print_world
    )
    task1 >> task2  # Task1 runs before Task2

Copy this HTML into a blog editor, and the Mermaid diagrams will render as interactive graphs. The Python code is styled with a light-gray background for readability. You’re all set!

Saturday, March 15, 2025

Introduction - FullStack Developer/ Data Engineer

Why I'm the Best Fit for This Role

Thank you for the briefing. Now, I can understand more about the position. After listening to the conversation, I am more confident to say that, “I am best fit for this role.”

Introduction

I am a Full-Stack Python Developer, with all years of experience, with around 80% into Backend work and 20% into the frontend.

Backend Expertise

In Backend, I worked on:

Creating standalone scripts for automation, scheduled jobs, ETL jobs, and data pipelines.
Building RESTful APIs and web applications.

Frontend Expertise

In Frontend, I worked on:

Creating dashboards, graphs, and charts using d3.js or Highcharts libraries.
Building tables using ag-grid.
Working with JavaScript, jQuery, and React.

Databases

In databases:

Relational Databases: MySQL, MS SQL, Oracle DB, PostgreSQL.
NoSQL Databases: MongoDB, Cassandra.
Data Warehousing: Snowflake schema.

Python Expertise

In Python:

Creating web applications and RESTful APIs using Django, Flask, and FastAPI frameworks.
Process automation and integrating with infrastructure (Linux/Windows).
Data gathering from:
1. Structured datasources like REST APIs (internal or external), databases, etc.
2. Unstructured datasources like web scraping using Beautiful Soup.
3. Structured, semi-structured, or unstructured file types like CSV, Excel, JSON, YAML, Parquet, etc.
Following TDD (Test-Driven Development) by creating unit tests and integration tests using unittest or pytest modules.

Caching and Scheduling

In caching:

Worked with Redis and Memcache.

In scheduling:

Worked with Celery.
Integrated Celery with Django applications for periodic jobs.

For data job orchestration, I worked with Airflow.

Public Cloud Experience

In Public Cloud, I am mostly associated with AWS Cloud. In AWS Cloud, I worked with:

Server-Based Architectures

EC2 Instances
Elastic Beanstalk
Elastic Load Balancer
Auto-Scaling
Route53

Storage Solutions

S3 Bucket for file storage
S3 Glacier for archival storage

Databases

AWS RDS & Redshift for relational databases
AWS DynamoDB for NoSQL

Serverless Architectures

AWS Lambda (time-triggered or event-triggered)
AWS API Gateway (HTTP-triggered)
AWS Event Scheduler for scheduling Lambda

Container-based Environments

Created docker.yaml files for creating containers
AWS EKS (Elastic Kuberntes Service) for Orchestrating the pods.
Also, wrote the helm chart

For OAuth, I worked with AWS Cognito.

Also worked with SQS, SNS, and SES.

For big data processing, I worked with EMR clusters for PySpark.

For ETLs, I worked with AWS Glue Jobs with DataCatalog and PySpark.

CI/CD and Infrastructure as Code

Experience in creating CI/CD setups using Jenkins and Groovy scripts.

In terms of Infrastructure as Code, I worked mainly with:

CloudFormation templates
Terraform
Pulumi Python module

Agile Methodologies

Experience in agile methodologies like Scrum and Kanban in facilitating agile ceremonies like:

Daily standups
Sprint reviews
Planning sessions
Retrospectives

Familiar with Agile tools like Jira and skilled in working with cross-functional teams including Dev teams, QA testers, and PM/POs.

Wednesday, September 6, 2023

Python Interview Questions

Python Interview Quiz

Difficulty: Topic:

1. What happens when you use the += operator on a list with a string in Python?
A) It appends each character of the string to the list.
B) It replaces the list with the string.
C) It concatenates the string to the list as a single element.
D) It raises a TypeError.

2. How do you use Abstract Base Classes (ABCs) in Python, and in what contexts are they suitable?
A) ABCs are used to handle exceptions.
B) ABCs are used to create concrete classes.
C) ABCs are used for metaprogramming.
D) ABCs define a common interface for subclasses to implement specific methods.

3. What are the results of 5/2, 5.0/2, and 5*1.0/2 in Python?
A) 2, 2, 2.5
B) 2.5, 2, 2.5
C) 2.5, 2.5, 2.5
D) 2, 2.5, 2.5

4. How are arguments passed by value or by reference in Python?
A) By value
B) By name
C) By reference
D) By object reference

5. How is Python an interpreted language?
A) Python is just-in-time compiled.
B) Python is interpreted line-by-line.
C) Python is compiled to machine code.
D) Python is transpiled to another language.

6. What is the purpose of PYTHONPATH, and how do you modify it?
A) PYTHONPATH is used to set the default encoding.
B) PYTHONPATH is used to set the Python interpreter's version.
C) PYTHONPATH is an environment variable for module search paths.
D) PYTHONPATH is used to define virtual environments.

7. What is a key difference between Python 2 and Python 3 regarding the print statement?
A) Python 2 uses print() while Python 3 uses printf().
B) In Python 3, print is a function and requires parentheses.
C) Python 2 doesn't support print statements.
D) There is no difference; print works the same in both.

8. How do you debug a Python program?
A) Use try-except blocks.
B) Use logging or traceback.
C) Use assert statements.
D) Use print statements or pdb.

9. What is the output of this code? d = {'k1': 'alpha'} d.update(key3='gamma') d.update({'k1': 'delta'}) print(d)
A) {'k1': 'alpha', 'key3': 'gamma'}
B) {'k1': 'alpha'}
C) {'k1': 'gamma', 'key3': 'delta'}
D) {'k1': 'delta', 'key3': 'gamma'}

10. Which pair best illustrates an abstract-concrete class relationship?
A) Cat - Kitten
B) Colour - Red
C) Truck - Window
D) Planet - Moon

11. How do you comment multiple lines in Python?
A) Use // for multi-line comments.
B) Use for comments.
C) Use /* */ for multi-line comments.
D) Use triple quotes or # on each line.

12. What is the output of this code?


def pair_the_items():
    item_stack = [1, 2, 3, 4]
    idx = 0
    item_pairs = zip(item_stack[idx:-1], item_stack[idx+1:])
    print(type(item_pairs))
    print(f'item pairs: {list(item_pairs)}')
pair_the_items()

A) item pairs: (1, 2, 3, 4)
B) item pairs: [(1, 2, 3, 4)]
C) item pairs: [(1, 2), (2, 3), (3, 4)]
D) item pairs: [(1, 2), (2, 3), (3, 4)]

13. Which statement is used to force a particular exception to occur in Python?
A) except
B) throws
C) throw
D) raise

14. Which utility is used to install or remove Python packages?
A) yum
B) sudo
C) pip
D) pep

15. How do you count the occurrences of each character in a string and return the result as a dictionary?
A) Use a list to store counts.
B) Use a tuple to store character frequencies.
C) Use a set to count occurrences.
D) Use a dictionary with setdefault or defaultdict.

16. What is the purpose of Python decorators, and how can they be used?
A) Decorators are used to extend function logic by handle redundant code among multiple functions by wrapping them.
B) Decorators are used to create anonymous functions.
C) Decorators are used to import functions and decorate them.
D) Decorators are used to define classes.

17. How do you remove values from a Python array?
A) Use del list[index] to remove by index.
B) All of the above.
C) Use list.pop(index) to remove by index.
D) Use list.remove(value) to remove by value.

18. What is wrong with the following code?


odd = lambda x: bool(x % 2)
numbers = [n for n in range(10) if odd() if not odd(n)]
for i in range(len(numbers)):
    if odd(numbers[i]):
        del numbers[i]
print(numbers)

A) The code is correct and will run without errors.
B) The list comprehension is correct, but deleting items while iterating causes index errors.
C) The lambda function is used incorrectly, and the list comprehension has a syntax error.
D) The list comprehension is incorrect, and the loop will cause a MemoryError.

19. What is the output of this code?


d1 = {}
d1['a'] = 1
d1['b'] = 2
d2 = d1
d2['a'] = 10
print(d1)
print(d2)

A) {'a': 10} {'a': 10}
B) {'a': 10, 'b': 2} {'a': 10, 'b': 2}
C) {'a': 1, 'b': 2} {'a': 10, 'b': 2}
D) {'a': 1, 'b': 2} {'a': 1, 'b': 2}

20. What is the output of this code? from string import * method = 'METHODS' def x(methods): method = str.swapcase(methods) print(('%(method)s' % locals())) methods = str.swapcase(method[:-1]) x(methods)
A) 'Methods'
B) 'methods'
C) 'METHOD'
D) 'method'

21. What is the output of the following code and why?


class Foo:
    def __init__(self):
        self.x = 5
        self.bubbles = "Bubbles"
    def buzz(self):
        print(self.bubbles)
f = Foo()
f.buzz()

A) True, because the list is modified in place.
B) True, because the function returns the id of a new list.
C) False, because the ids are different.
D) False, because a new list is created.

22. What are list and dictionary comprehensions in Python? How do you create them and filter items?
A) Comprehensions provide a concise way to create lists or dictionaries with optional filtering.
B) Comprehensions are used to define classes.
C) Comprehensions are used to create functions.
D) Comprehensions are used for exception handling.

23. Which statement best describes the Memento and Observer design patterns?
A) Memento alters an object's behavior when its state changes. Observer encapsulates an algorithm inside a class.
B) Memento defers the exact steps of an algorithm to a subclass. Observer defines a new operation.
C) Memento captures and restores an object's internal state, whereas Observer notifies multiple classes of changes.

24. What is the output of the following code?


class Foo(object):
    def __init__(self):
        self.x = 5
        self.bubbles = 2
class Buzz(Foo):
    def __init__(self):
        Foo.__init__(self)
        self.bubbles = 'Bubbles'
foo = Foo()
print(foo.x)
print(foo.bubbles)
buzz = Buzz()
print(buzz.x)
print(buzz.bubbles)
foo.x = 22
print(buzz.x)
print(Foo.bubbles)
Foo.bubbles = 'dasf'
print(foo.bubbles)
print(foo.x)

A) 5 2 5 'Bubbles' 22 2 'dasf' 22
B) 5 2 5 'Bubbles' 5 AttributeError
C) 5 2 5 'Bubbles' 22 AttributeError
D) 5 2 5 'Bubbles' 5 2 'dasf' 22

25. What does the `lambda` keyword do in Python?
A) Defines a named function.
B) Imports a module.
C) Declares a loop.
D) Creates an anonymous function.

26. What is a generator in Python, and where might you use one?
A) Generators yield values one at a time and are memory-efficient.
B) Generators are used for parallel processing.
C) Generators are used for creating lists.
D) Generators are used for exception handling.

27. How do you insert an object at a given index in Python?
A) Use list.append()
B) Use list.insert(index, object)
C) Use list.add(index, object)
D) Use list[index] = object

28. How do you create a nested comprehension in Python, and how does it compare to traditional for loops?
A) Nested comprehensions are used for exception handling.
B) Nested comprehensions use multiple for loops in a single statement and are more concise but can be less readable.
C) Nested comprehensions are not supported in Python.
D) Nested comprehensions are less concise than for loops.

29. How do you copy an object in Python?
A) Use object.copy()
B) Use deepcopy() only
C) Use clone()
D) Use copy.copy() or copy.deepcopy()

30. What is the value of z after this code? x = 1 y = 0 z = 0 result = 2 try: z = z + 1 result = x / y z = z + 1 except ZeroDivisionError: z = z + result else: z = z + 1 finally: z = z + 1 z = z + 1 print(z)
A) 3
B) 6
C) 5
D) 4

31. How do you get the current working directory in Python?
A) import os.path; os.path.getcwd()
B) import sys; sys.getcwd()
C) import os; os.getcwd()
D) import pwd; pwd()

32. How do you install Python on Windows and set the path variable?
A) Use pip to install Python.
B) Download from GitHub and manually set the path.
C) Download from python.org and check 'Add Python to PATH'.
D) Install via Anaconda.

33. Does Python have OOP concepts?
A) Python only supports procedural programming.
B) Yes, Python supports OOP concepts.
C) Python supports OOP but not inheritance.
D) No, Python does not support OOP concepts.

34. What happens when you append to a list while iterating over it in Python?
A) It raises a RuntimeError.
B) It removes elements instead of appending.
C) It works fine and appends elements correctly.
D) It can lead to infinite loops or errors.

35. How is memory managed in Python?
A) Python does not manage memory.
B) Python uses reference counting and garbage collection.
C) Python uses manual memory management.
D) Python uses a mark-and-sweep algorithm.

36. How do you sort a list of dictionaries by multiple keys, such as 'lastName' and then 'firstName'?
A) sorted(my_list, key=lambda d: d['lastName'] + d['firstName'])
B) my_list.sort(key=lambda d: d['lastName'], d['firstName'])
C) sorted(my_list, key=lambda d: (d['lastName'], d['firstName']))
D) sorted(my_list, key=[d['lastName'], d['firstName']])

37. What is the output of sorting this array alphabetically by food name and then by number?


foods = ['Apple-1', 'Cucumber-5', 'Apple-33', 'Banana-17', 'Strawberry-15', 'Strawberry-4', 'Apple-4', 'Coconut-7', 'Banana-18', 'Strawberry-23']

A) ['Apple-1', 'Apple-4', 'Apple-33', 'Banana-17', 'Banana-18', 'Coconut-7', 'Cucumber-5', 'Strawberry-4', 'Strawberry-15', 'Strawberry-23']
B) ['Apple-1', 'Apple-33', 'Apple-4', 'Banana-17', 'Banana-18', 'Coconut-7', 'Cucumber-5', 'Strawberry-15', 'Strawberry-23', 'Strawberry-4']
C) ['Cucumber-5', 'Coconut-7', 'Banana-18', 'Banana-17', 'Apple-33', 'Apple-4', 'Apple-1', 'Strawberry-23', 'Strawberry-15', 'Strawberry-4']
D) ['Strawberry-4', 'Strawberry-15', 'Strawberry-23', 'Coconut-7', 'Cucumber-5', 'Banana-17', 'Banana-18', 'Apple-1', 'Apple-4', 'Apple-33']

38. How do you sort a list of dictionaries by 'lastName' in place?
A) my_list.sort(key=d['lastName'])
B) sorted(my_list, key=lambda d: d['lastName'])
C) my_list.sort(key=lambda d: d['lastName'])
D) sorted(my_list, key=d['lastName'])

39. What is the output of this code?


list1 = [1, 3, 5, 7, 9]
list2 = [0, 2, 4, 6, 8]
print({x: y for x in list1 for y in list2})

A) {1: 8, 3: 8, 5: 8, 7: 8, 9: 8}
B) {0: 1, 2: 3, 4: 5, 6: 7, 8: 9}
C) {1: 0, 3: 2, 5: 4, 7: 6, 9: 8}
D) {1: [0, 2, 4, 6, 8], 3: [0, 2, 4, 6, 8], ...}

40. What is the purpose of *args and **kwargs in Python?
A) *args is for default arguments, **kwargs for variable arguments.
B) *args is for keyword arguments, **kwargs for positional arguments.
C) *args is for named arguments, **kwargs for unnamed arguments.
D) *args is for positional arguments, **kwargs for keyword arguments.

41. How do you remove whitespaces from a string in Python?
A) Use string.lstrip() and string.rstrip()
B) Use string.replace(' ', '')
C) All of the above
D) Use string.strip()

42. What is the output of `[x for x in range(5) if x % 2 == 0]`?
A) [0, 1, 2, 3, 4]
B) [2, 4]
C) [0, 2, 4]
D) [1, 3]

43. What will be the output of `3 / 2` in Python 3?
A) 1
B) Error
C) 2
D) 1.5

44. What is the output of this code?


lst1 = [1, 2, 3]
lst2 = lst1
lst3 = lst2
del lst2[1:2]
print(lst3)

A) [1, 3]
B) [2, 3]
C) [1, 2, 3]
D) []

45. What are Python functions?
A) Functions are used for looping.
B) Functions are used for creating classes.
C) Functions are used for exception handling.
D) Functions are reusable blocks of code that perform a specific task.

46. How do you remove leading whitespaces from a string in Python?
A) Use string.replace(' ', '')
B) Use string.lstrip()
C) Use string.rstrip()
D) Use string.strip()

47. In Python's interactive mode, which variable holds the value of the last printed expression?
A) _
B) -
C) *
D) ?

48. How can you generate random numbers in Python?
A) Use the random module.
B) Use the numpy module.
C) Use the math module.
D) Use the statistics module.

49. Which of these is NOT a valid way to handle exceptions in Python?
A) try-except
B) catch-throw
C) try-except-else
D) try-finally

50. How can we make forms in Python?
A) Use SQLAlchemy for forms.
B) Use Django or Flask for web forms, Tkinter for GUI forms.
C) Use PyQt for web forms, Django for GUI forms.
D) Use Flask for GUI forms, Tkinter for web forms.

51. How can you set up a database in Django?
A) Edit admin.py and run 'python manage.py createdb'
B) Edit models.py and run 'python manage.py syncdb'
C) Edit settings.py and run 'python manage.py migrate'
D) Edit urls.py and run 'python manage.py makemigrations'

52. Given an array of 99 unique numbers from 1 to 100, how do you find the missing number, and what is the run time?
A) Use a hash set to find the missing number in O(n) time.
B) Sort the array and find the missing number in O(n log n) time.
C) Subtract the sum of the array from 5050 in O(n) time.
D) Use binary search to find the missing number in O(log n) time.

53. What is the difference in variable scope between Python 2 and Python 3 in list comprehensions?
A) In Python 2, list comprehensions leak variables; in Python 3, they do not.
B) List comprehensions are not supported in Python 2.
C) There is no difference in variable scope between Python 2 and Python 3.
D) In Python 2, list comprehensions do not leak variables; in Python 3, they do.

54. What is the value of z after this code? z = 5 x = '456' while z > -1: y = x if y in ('1', '2', '3') or len(y) > 1: break x = x + y if y in ('4', '5', '6'): continue z = z - 1 print(z)
A) -1
B) 0
C) 10
D) 5

55. What is the default secondary prompt for continuation lines in Python's interactive mode?
A) ...
B) ..
C) >>
D) >>>

56. What is the purpose of the property decorator in Python?
A) The property decorator allows methods to be accessed as attributes.
B) The property decorator is used to define static methods.
C) The property decorator is used to create class methods.
D) The property decorator is used to handle exceptions.

57. How do you delete a file in Python?
A) Use os.remove('filename')
B) Use os.delete('filename')
C) Use shutil.remove('filename')
D) Use file.delete('filename')

58. What is the output of this code? a = {1, 2, 3, 4} b = {2, 3, 4, 5} c = {3, 4, 5, 6} d = {4, 5, 6, 7} v = a.union(b, c) print(v | d)
A) {1, 2, 3, 4}
B) {4, 5, 6, 7}
C) {1, 2, 3, 4, 5, 6}
D) {1, 2, 3, 4, 5, 6, 7}

59. What is the difference between a list and a tuple in Python?
A) Lists are mutable, tuples are immutable.
B) Both are mutable.
C) Lists are immutable, tuples are mutable.
D) Both are immutable.

60. Which expression represents 10 to the power of 3 in Python?
A) 10 * 3
B) 10 ^ 3
C) 10 ** 3
D) 10 (3)

61. How does Python's garbage collection work, and what is the `gc` module?
A) Python uses mark-and-sweep garbage collection.
B) Python uses a stop-the-world garbage collector.
C) Python does not have garbage collection.
D) Python uses reference counting and a cyclic garbage collector.

62. What is a reference to an object in Python?
A) It is the address of a variable only; not the methods of an object.
B) It is the address where the variables and methods of an object are stored.
C) It is a shallow pointer that contains the address of an object.
D) It is the physical address of an object.

63. What happens when you multiply a list containing a sublist and modify it?
A) Each sublist is independent, so modifying one does not affect others.
B) List multiplication creates deep copies of the sublist.
C) All sublists are references to the same list, so modifying one affects all.
D) Modifying a sublist will raise an error.

64. How do you create a Unicode string in Python?
A) Use 'text'.encode('utf-8')
B) Use u'text' in Python 3.
C) All strings are Unicode in Python 3.
D) Use b'text' for Unicode strings.

65. What is the output of print(name[-1]) if name = 'Python'?
A) Invalid Syntax
B) End-of-lines
C) n
D) P

66. Which built-in types in Python are mutable and which are immutable?
A) Mutable: dict, set, tuple; Immutable: list, frozenset, str
B) Mutable: list, dict, tuple; Immutable: set, frozenset, str
C) Mutable: list, dict, set; Immutable: tuple, frozenset, str
D) Mutable: list, tuple, set; Immutable: dict, frozenset, str

67. What happens when you access x[3] in a list with only 3 elements?
A) It wraps around to the first element.
B) It returns None.
C) It returns the last element.
D) It raises an IndexError.

Friday, June 5, 2020

getattrib, Setattrib, hasattrib and delattrib in python

# `getattr(object, name[, default])` Function in Python

The `getattr(object, name[, default])` function returns the value of a named attribute of an object, where `name` must be a string. If the object has an attribute with the specified `name`, then the value of that attribute is returned. On the other hand, if the object does not have an attribute with `name`, then the value of `default` is returned, or `AttributeError` is raised if `default` is not provided.

```python

>>> t = ('This', 'is', 'a', 'tuple')

>>> t.index('is')

>>> getattr(t, 'index')

<built-in method index of tuple object at 0x10c15e680>

>>> getattr(t, 'index')('is')

```

when the attribute is not defined,

```python

>>> getattr(t, 'len')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'tuple' object has no attribute 'len'
>>> getattr(t, 'len', t.count)
<built-in method count of tuple object at 0x10c15e680>

```

Besides "normal" objects like tuples, lists, and class instances, getattr also accepts modules as arguments. Since modules are also objects in Python, the attributes of modules can be retrieved just like any attribute in an object.
>>> import uuid
>>> getattr(uuid, 'UUID')
<class 'uuid.UUID'>
>>> type(getattr(uuid, 'UUID'))
<type 'type'>
>>> isinstance(getattr(uuid, 'UUID'), type)
True
>>> callable(getattr(uuid, 'UUID'))
True
>>> getattr(uuid, 'UUID')('12345678123456781234567812345678')
UUID('12345678-1234-5678-1234-567812345678')

*****

# check for existence of an attribute
# hasattr #uses pythonic "Look Before You Leap" (LBYL) code style

>>> hasattr('abc', 'upper')
True
>>> hasattr('abc', 'lower')
True
>>> hasattr('abc', 'convert')
False

Using try-except way # "Easier to Ask for Forgiveness than Permission" (EAFP) code style

>>> try:
...     'abc'.upper()
... except AttributeError:
...     print("abc does not have attribute 'upper'")
...
'ABC'
>>> try:
...     'abc'.convert()
... except AttributeError:
...     print("abc does not have attribute 'convert'")
...
abc does not have attribute 'convert'

**********

hasattr vs dict

>>> class A(object):
...   foo = 1
...
>>> class B(A):
...   pass
...
>>> b = B()
>>> hasattr(b, 'foo')
True
>>> 'foo' in b.__dict__
False

#setattrib --Assigns a value to the object’s attribute given its name.

EX: setattr(x, ‘foobar’, 123) is equivalent to x.foobar = 123

Example 1

>>> class Foo:
...     def __init__(self, x):
...         self.x = x
...
>>> f = Foo(10)
>>> f.x
10
>>> setattr(f, 'x', 20)
>>> f.x
20
>>> setattr(f, 'y', 10)
>>> f.y
10
>>> f.y = 100
>>> f.y
100

you can dynamically add a function as a method to a class

>>> def b(self):
            print 'bar'

class Foo:
    pass
f = Foo()
print dir(f) #[‘__doc__’, ‘__module__’]
setattr(Foo, ‘bar’, b)
print dir(f) #[‘__doc__’, ‘__module__’, ‘bar’]
f.bar() #bar

you can setattr() on an instance of a class that
inherits from "object", but you can't on an instance of "object"
itself

>>> o = object()
>>> setattr(o, "x", 1000)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'object' object has no attribute 'x'

>>> class Object(object):pass
...
>>> o = Object()
>>> setattr(o, "x", 1000)
>>> o.x
1000

If __setattr__() wants to assign to an instance attribute,
it should not simply execute self.name = value — this would
cause a recursive call to itself. Instead, it should insert
the value in the dictionary of instance attributes,
e.g., self.__dict__[name] = value.
For new-style classes, rather than accessing the instance dictionary,
it should call the base class method with the same name,
for example, object.__setattr__(self, name, value).

****for blog****
The __slots__ declaration takes a sequence of instance variables and
reserves just enough space in each instance to hold a value for each
variable. Space is saved because __dict__ is not created for each instance.

When there is '__slots__', there won't be '__dict__' and '__weakref__'

delattr will delete the attribute

Example:

class Box:
    pass

box = Box()

# Create a width attribute.
setattr(box, "width", 15)

# The attribute exists.
if hasattr(box, "width"):
    print(True)

# Delete the width attribute.
delattr(box, "width")

# Width no longer exists.
if not hasattr(box, "width"):
    print(False)

Python tips

python tips:

In [1]: os.listdir(os.getcwd())==os.listdir(os.curdir)

Out[1]: True

In [2]: os.listdir(os.getcwd())==os.listdir('.')

Out[2]: True

*******
WAP for print a character its position times in a word.

word=raw_input("Enter the word:")

for i in word:
    print i*word.index(i)

Output:

******
WAP for print a character its ASCII values times in a word.



word=raw_input("Enter the word:")

for i in word:
 print i*ord(i)

#alternatively
# print [i*ord(i) for i in word]

Sunday, December 17, 2017

Initiating static local HTTP Server using different scripting languages

Initiating HTTP Server using different scripting languages

Python

python 2.x - python -m SimpleHTTPServer 8000

python 3.x - python -m http.server 8000

http-server (Node.js)

npm install -g http-server # install dependency

http-server -p 8000

node-static (Node.js)

npm install -g node-static # install dependency

static -p 8000

Ruby

ruby -rwebrick -e’WEBrick::HTTPServer.new(:Port => 3000, :DocumentRoot => Dir.pwd).start’

Ruby 1.9.2+

- ruby -run -ehttpd . -p3000

adsf (Ruby)

- gem install adsf # install dependency

- adsf -p 8000

Perl

----

cpan HTTP::Server::Brick # install dependency

- $ perl -MHTTP::Server::Brick -e ‘$s=HTTP::Server::Brick->new(port=>8000); $s->mount(“/”=>{path=>”.”}); $s->start’

Plack (Perl)

- cpan Plack # install dependency

- plackup -MPlack::App::Directory -e ‘Plack::App::Directory->new(root=>”.”);’ -p 8000

PHP (>= 5.4)

------------

php -S 127.0.0.1:8000

busybox httpd

-------------

busybox httpd -f -p 8000

webfs

-----

webfsd -F -p 8000

IIS Express

-----------

C:\> “C:\Program Files (x86)\IIS Express\iisexpress.exe” /path:C:\MyWeb /port:8000

AWS CLI installation procedure in centos/ RHEL

1) Make sure you have the epel repository installed
sudo yum install -y epel-release

2) Install needed packages
sudo yum install -y python2-pip

3) Install awscli
sudo yum install -y awscli

4) Configure awscli
aws configure --profile <profile_name>

4) Configure awscli.
If you do not add --profile <profile_name>, it will use the default profile.

aws configure --profile <profile_name>

Using the --profile allows you to create profiles for other accounts using different keys.
You can also create profiles for different regions.

Saturday, March 22, 2025

Apache Airflow: A One-Stop Guide for Junior Developers

History and Evolution

What is Airflow?

Architecture with Mermaid Diagram

Executor Modes with Mermaid Diagrams

LocalExecutor

CeleryExecutor

Sample Python DAG Code

Saturday, March 15, 2025

Introduction

Backend Expertise

Frontend Expertise

Databases

Python Expertise

Caching and Scheduling

Public Cloud Experience

Server-Based Architectures

Storage Solutions

Databases

Serverless Architectures

Container-based Environments

CI/CD and Infrastructure as Code

Agile Methodologies

Wednesday, September 6, 2023

Python Interview Quiz

Friday, June 5, 2020

hasattr vs __dict__

Sunday, December 17, 2017

hasattr vs dict