Udhay Prakash Blog

Saturday, March 22, 2025

Apache Airflow notes

Apache Airflow: A One-Stop Guide for Junior Developers

Apache Airflow is a powerful open-source tool for orchestrating jobs and managing data workflows. This guide covers everything you need—history, features, and a practical example—all explained simply.

History and Evolution

Workflow orchestration evolved over time. Here’s the journey:

Non/Quasi-Programmable Tools (e.g., Informatica, Talend):
Early tools like Informatica and Talend offered graphical interfaces for ETL workflows. While powerful for simple tasks, they weren’t fully programmable, limiting flexibility, dependency management, and version control.
cronTab and Event Scheduler:
Basic scheduling tools like cronTab (Linux) and Event Scheduler (Windows) ran jobs at fixed times but couldn’t handle dependencies or track job status.
Celery:
A step up, Celery provided a task queue with workers but required custom logic for workflows.
Apache Airflow (2014):
Created at Airbnb in 2014 and open-sourced in 2015, Airflow introduced code-defined workflows with dependency management, becoming an Apache project in 2016.

What is Airflow?

Airflow lets you programmatically define, schedule, and monitor workflows using Python. Workflows are represented as Directed Acyclic Graphs (DAGs)—tasks with a defined order and no loops.

Architecture with Mermaid Diagram

Here’s how Airflow’s components connect:

Web Server: Hosts the UI for monitoring.
Scheduler: Schedules tasks based on DAGs.
Executor: Manages task execution.
Workers: Run the tasks.
Metadata Database: Stores task states and logs.

Executor Modes with Mermaid Diagrams

LocalExecutor

Runs tasks on the same machine as the scheduler—simple but not scalable.

CeleryExecutor

Distributes tasks across workers using Celery—scalable but needs a broker like Redis.

Sample Python DAG Code

Here’s a styled Python DAG example that prints "Hello" and "World!":

from airflow import DAG
from airflow.operators.python import PythonOperator
from datetime import datetime

def print_hello():
    print("Hello")

def print_world():
    print("World!")

with DAG(
    dag_id='hello_world_dag',
    start_date=datetime(2023, 1, 1),
    schedule_interval='@daily',
    catchup=False
) as dag:
    task1 = PythonOperator(
        task_id='print_hello',
        python_callable=print_hello
    )
    task2 = PythonOperator(
        task_id='print_world',
        python_callable=print_world
    )
    task1 >> task2  # Task1 runs before Task2

Copy this HTML into a blog editor, and the Mermaid diagrams will render as interactive graphs. The Python code is styled with a light-gray background for readability. You’re all set!

Saturday, March 15, 2025

Introduction - FullStack Developer/ Data Engineer

Why I'm the Best Fit for This Role

Thank you for the briefing. Now, I can understand more about the position. After listening to the conversation, I am more confident to say that, “I am best fit for this role.”

Introduction

I am a Full-Stack Python Developer, with all years of experience, with around 80% into Backend work and 20% into the frontend.

Backend Expertise

In Backend, I worked on:

Creating standalone scripts for automation, scheduled jobs, ETL jobs, and data pipelines.
Building RESTful APIs and web applications.

Frontend Expertise

In Frontend, I worked on:

Creating dashboards, graphs, and charts using d3.js or Highcharts libraries.
Building tables using ag-grid.
Working with JavaScript, jQuery, and React.

Databases

In databases:

Relational Databases: MySQL, MS SQL, Oracle DB, PostgreSQL.
NoSQL Databases: MongoDB, Cassandra.
Data Warehousing: Snowflake schema.

Python Expertise

In Python:

Creating web applications and RESTful APIs using Django, Flask, and FastAPI frameworks.
Process automation and integrating with infrastructure (Linux/Windows).
Data gathering from:
1. Structured datasources like REST APIs (internal or external), databases, etc.
2. Unstructured datasources like web scraping using Beautiful Soup.
3. Structured, semi-structured, or unstructured file types like CSV, Excel, JSON, YAML, Parquet, etc.
Following TDD (Test-Driven Development) by creating unit tests and integration tests using unittest or pytest modules.

Caching and Scheduling

In caching:

Worked with Redis and Memcache.

In scheduling:

Worked with Celery.
Integrated Celery with Django applications for periodic jobs.

For data job orchestration, I worked with Airflow.

Public Cloud Experience

In Public Cloud, I am mostly associated with AWS Cloud. In AWS Cloud, I worked with:

Server-Based Architectures

EC2 Instances
Elastic Beanstalk
Elastic Load Balancer
Auto-Scaling
Route53

Storage Solutions

S3 Bucket for file storage
S3 Glacier for archival storage

Databases

AWS RDS & Redshift for relational databases
AWS DynamoDB for NoSQL

Serverless Architectures

AWS Lambda (time-triggered or event-triggered)
AWS API Gateway (HTTP-triggered)
AWS Event Scheduler for scheduling Lambda

Container-based Environments

Created docker.yaml files for creating containers
AWS EKS (Elastic Kuberntes Service) for Orchestrating the pods.
Also, wrote the helm chart

For OAuth, I worked with AWS Cognito.

Also worked with SQS, SNS, and SES.

For big data processing, I worked with EMR clusters for PySpark.

For ETLs, I worked with AWS Glue Jobs with DataCatalog and PySpark.

CI/CD and Infrastructure as Code

Experience in creating CI/CD setups using Jenkins and Groovy scripts.

In terms of Infrastructure as Code, I worked mainly with:

CloudFormation templates
Terraform
Pulumi Python module

Agile Methodologies

Experience in agile methodologies like Scrum and Kanban in facilitating agile ceremonies like:

Daily standups
Sprint reviews
Planning sessions
Retrospectives

Familiar with Agile tools like Jira and skilled in working with cross-functional teams including Dev teams, QA testers, and PM/POs.

Wednesday, September 6, 2023

Python Interview Questions

Python Interview Quiz

Difficulty: Topic:

1. How do you remove whitespaces from a string in Python?
A) Use string.strip()
B) All of the above
C) Use string.lstrip() and string.rstrip()
D) Use string.replace(' ', '')

2. How do you use Abstract Base Classes (ABCs) in Python, and in what contexts are they suitable?
A) ABCs are used to create concrete classes.
B) ABCs define a common interface for subclasses to implement specific methods.
C) ABCs are used to handle exceptions.
D) ABCs are used for metaprogramming.

3. How do you create a nested comprehension in Python, and how does it compare to traditional for loops?
A) Nested comprehensions use multiple for loops in a single statement and are more concise but can be less readable.
B) Nested comprehensions are less concise than for loops.
C) Nested comprehensions are not supported in Python.
D) Nested comprehensions are used for exception handling.

4. How does Python's garbage collection work, and what is the `gc` module?
A) Python does not have garbage collection.
B) Python uses reference counting and a cyclic garbage collector.
C) Python uses a stop-the-world garbage collector.
D) Python uses mark-and-sweep garbage collection.

5. Which expression represents 10 to the power of 3 in Python?
A) 10 ^ 3
B) 10 (3)
C) 10 * 3
D) 10 ** 3

6. Which utility is used to install or remove Python packages?
A) pep
B) yum
C) sudo
D) pip

7. What is the purpose of PYTHONPATH, and how do you modify it?
A) PYTHONPATH is used to set the default encoding.
B) PYTHONPATH is an environment variable for module search paths.
C) PYTHONPATH is used to define virtual environments.
D) PYTHONPATH is used to set the Python interpreter's version.

8. What are list and dictionary comprehensions in Python? How do you create them and filter items?
A) Comprehensions are used to define classes.
B) Comprehensions are used to create functions.
C) Comprehensions provide a concise way to create lists or dictionaries with optional filtering.
D) Comprehensions are used for exception handling.

9. What is a key difference between Python 2 and Python 3 regarding the print statement?
A) In Python 3, print is a function and requires parentheses.
B) There is no difference; print works the same in both.
C) Python 2 uses print() while Python 3 uses printf().
D) Python 2 doesn't support print statements.

10. Which statement best describes the Memento and Observer design patterns?
A) Memento alters an object's behavior when its state changes. Observer encapsulates an algorithm inside a class.
B) Memento defers the exact steps of an algorithm to a subclass. Observer defines a new operation.
C) Memento captures and restores an object's internal state, whereas Observer notifies multiple classes of changes.

11. Which built-in types in Python are mutable and which are immutable?
A) Mutable: dict, set, tuple; Immutable: list, frozenset, str
B) Mutable: list, dict, set; Immutable: tuple, frozenset, str
C) Mutable: list, dict, tuple; Immutable: set, frozenset, str
D) Mutable: list, tuple, set; Immutable: dict, frozenset, str

12. Given an array of 99 unique numbers from 1 to 100, how do you find the missing number, and what is the run time?
A) Subtract the sum of the array from 5050 in O(n) time.
B) Use a hash set to find the missing number in O(n) time.
C) Use binary search to find the missing number in O(log n) time.
D) Sort the array and find the missing number in O(n log n) time.

13. How do you comment multiple lines in Python?
A) Use // for multi-line comments.
B) Use /* */ for multi-line comments.
C) Use for comments.
D) Use triple quotes or # on each line.

14. How do you sort a list of dictionaries by 'lastName' in place?
A) sorted(my_list, key=lambda d: d['lastName'])
B) sorted(my_list, key=d['lastName'])
C) my_list.sort(key=lambda d: d['lastName'])
D) my_list.sort(key=d['lastName'])

15. What is the output of the following code?


class Foo(object):
    def __init__(self):
        self.x = 5
        self.bubbles = 2
class Buzz(Foo):
    def __init__(self):
        Foo.__init__(self)
        self.bubbles = 'Bubbles'
foo = Foo()
print(foo.x)
print(foo.bubbles)
buzz = Buzz()
print(buzz.x)
print(buzz.bubbles)
foo.x = 22
print(buzz.x)
print(Foo.bubbles)
Foo.bubbles = 'dasf'
print(foo.bubbles)
print(foo.x)

A) 5 2 5 'Bubbles' 5 2 'dasf' 22
B) 5 2 5 'Bubbles' 5 AttributeError
C) 5 2 5 'Bubbles' 22 2 'dasf' 22
D) 5 2 5 'Bubbles' 22 AttributeError

16. What is the output of this code?


d1 = {}
d1['a'] = 1
d1['b'] = 2
d2 = d1
d2['a'] = 10
print(d1)
print(d2)

A) {'a': 10, 'b': 2} {'a': 10, 'b': 2}
B) {'a': 10} {'a': 10}
C) {'a': 1, 'b': 2} {'a': 10, 'b': 2}
D) {'a': 1, 'b': 2} {'a': 1, 'b': 2}

17. How is memory managed in Python?
A) Python uses reference counting and garbage collection.
B) Python does not manage memory.
C) Python uses a mark-and-sweep algorithm.
D) Python uses manual memory management.

18. How do you remove leading whitespaces from a string in Python?
A) Use string.lstrip()
B) Use string.rstrip()
C) Use string.strip()
D) Use string.replace(' ', '')

19. Does Python have OOP concepts?
A) Yes, Python supports OOP concepts.
B) Python supports OOP but not inheritance.
C) No, Python does not support OOP concepts.
D) Python only supports procedural programming.

20. Which statement is used to force a particular exception to occur in Python?
A) throws
B) throw
C) except
D) raise

21. What is a reference to an object in Python?
A) It is a shallow pointer that contains the address of an object.
B) It is the physical address of an object.
C) It is the address where the variables and methods of an object are stored.
D) It is the address of a variable only; not the methods of an object.

22. What is the output of this code?


def pair_the_items():
    item_stack = [1, 2, 3, 4]
    idx = 0
    item_pairs = zip(item_stack[idx:-1], item_stack[idx+1:])
    print(type(item_pairs))
    print(f'item pairs: {list(item_pairs)}')
pair_the_items()

A) item pairs: [(1, 2), (2, 3), (3, 4)]
B) item pairs: [(1, 2), (2, 3), (3, 4)]
C) item pairs: (1, 2, 3, 4)
D) item pairs: [(1, 2, 3, 4)]

23. How do you debug a Python program?
A) Use assert statements.
B) Use try-except blocks.
C) Use print statements or pdb.
D) Use logging or traceback.

24. Which of these is NOT a valid way to handle exceptions in Python?
A) try-except
B) catch-throw
C) try-except-else
D) try-finally

25. What is wrong with the following code?


odd = lambda x: bool(x % 2)
numbers = [n for n in range(10) if odd() if not odd(n)]
for i in range(len(numbers)):
    if odd(numbers[i]):
        del numbers[i]
print(numbers)

A) The code is correct and will run without errors.
B) The list comprehension is correct, but deleting items while iterating causes index errors.
C) The lambda function is used incorrectly, and the list comprehension has a syntax error.
D) The list comprehension is incorrect, and the loop will cause a MemoryError.

26. How do you remove values from a Python array?
A) Use list.pop(index) to remove by index.
B) All of the above.
C) Use list.remove(value) to remove by value.
D) Use del list[index] to remove by index.

27. What are the results of 5/2, 5.0/2, and 5*1.0/2 in Python?
A) 2.5, 2, 2.5
B) 2.5, 2.5, 2.5
C) 2, 2.5, 2.5
D) 2, 2, 2.5

28. What is the output of this code?


list1 = [1, 3, 5, 7, 9]
list2 = [0, 2, 4, 6, 8]
print({x: y for x in list1 for y in list2})

A) {1: 8, 3: 8, 5: 8, 7: 8, 9: 8}
B) {0: 1, 2: 3, 4: 5, 6: 7, 8: 9}
C) {1: [0, 2, 4, 6, 8], 3: [0, 2, 4, 6, 8], ...}
D) {1: 0, 3: 2, 5: 4, 7: 6, 9: 8}

29. What is the output of this code? from string import * method = 'METHODS' def x(methods): method = str.swapcase(methods) print(('%(method)s' % locals())) methods = str.swapcase(method[:-1]) x(methods)
A) 'Methods'
B) 'methods'
C) 'method'
D) 'METHOD'

30. What is the value of z after this code? x = 1 y = 0 z = 0 result = 2 try: z = z + 1 result = x / y z = z + 1 except ZeroDivisionError: z = z + result else: z = z + 1 finally: z = z + 1 z = z + 1 print(z)
A) 5
B) 4
C) 6
D) 3

31. What are Python functions?
A) Functions are used for creating classes.
B) Functions are used for exception handling.
C) Functions are used for looping.
D) Functions are reusable blocks of code that perform a specific task.

32. What happens when you append to a list while iterating over it in Python?
A) It can lead to infinite loops or errors.
B) It works fine and appends elements correctly.
C) It raises a RuntimeError.
D) It removes elements instead of appending.

33. What is the output of print(name[-1]) if name = 'Python'?
A) Invalid Syntax
B) End-of-lines
C) P
D) n

34. What is the output of this code? a = {1, 2, 3, 4} b = {2, 3, 4, 5} c = {3, 4, 5, 6} d = {4, 5, 6, 7} v = a.union(b, c) print(v | d)
A) {1, 2, 3, 4, 5, 6, 7}
B) {1, 2, 3, 4}
C) {4, 5, 6, 7}
D) {1, 2, 3, 4, 5, 6}

35. How do you get the current working directory in Python?
A) import os.path; os.path.getcwd()
B) import pwd; pwd()
C) import sys; sys.getcwd()
D) import os; os.getcwd()

36. What is the output of this code? d = {'k1': 'alpha'} d.update(key3='gamma') d.update({'k1': 'delta'}) print(d)
A) {'k1': 'alpha'}
B) {'k1': 'delta', 'key3': 'gamma'}
C) {'k1': 'alpha', 'key3': 'gamma'}
D) {'k1': 'gamma', 'key3': 'delta'}

37. How do you create a Unicode string in Python?
A) All strings are Unicode in Python 3.
B) Use b'text' for Unicode strings.
C) Use u'text' in Python 3.
D) Use 'text'.encode('utf-8')

38. What is the difference in variable scope between Python 2 and Python 3 in list comprehensions?
A) In Python 2, list comprehensions leak variables; in Python 3, they do not.
B) List comprehensions are not supported in Python 2.
C) In Python 2, list comprehensions do not leak variables; in Python 3, they do.
D) There is no difference in variable scope between Python 2 and Python 3.

39. What is the default secondary prompt for continuation lines in Python's interactive mode?
A) >>
B) >>>
C) ...
D) ..

40. What is the output of `[x for x in range(5) if x % 2 == 0]`?
A) [0, 1, 2, 3, 4]
B) [1, 3]
C) [2, 4]
D) [0, 2, 4]

41. How do you copy an object in Python?
A) Use object.copy()
B) Use copy.copy() or copy.deepcopy()
C) Use clone()
D) Use deepcopy() only

42. How do you sort a list of dictionaries by multiple keys, such as 'lastName' and then 'firstName'?
A) sorted(my_list, key=[d['lastName'], d['firstName']])
B) sorted(my_list, key=lambda d: (d['lastName'], d['firstName']))
C) my_list.sort(key=lambda d: d['lastName'], d['firstName'])
D) sorted(my_list, key=lambda d: d['lastName'] + d['firstName'])

43. What happens when you access x[3] in a list with only 3 elements?
A) It raises an IndexError.
B) It wraps around to the first element.
C) It returns the last element.
D) It returns None.

44. How is Python an interpreted language?
A) Python is interpreted line-by-line.
B) Python is transpiled to another language.
C) Python is just-in-time compiled.
D) Python is compiled to machine code.

45. In Python's interactive mode, which variable holds the value of the last printed expression?
A) -
B) _
C) *
D) ?

46. What is a generator in Python, and where might you use one?
A) Generators are used for exception handling.
B) Generators are used for parallel processing.
C) Generators yield values one at a time and are memory-efficient.
D) Generators are used for creating lists.

47. What does the `lambda` keyword do in Python?
A) Creates an anonymous function.
B) Defines a named function.
C) Declares a loop.
D) Imports a module.

48. What happens when you use the += operator on a list with a string in Python?
A) It appends each character of the string to the list.
B) It replaces the list with the string.
C) It raises a TypeError.
D) It concatenates the string to the list as a single element.

49. How do you count the occurrences of each character in a string and return the result as a dictionary?
A) Use a list to store counts.
B) Use a tuple to store character frequencies.
C) Use a dictionary with setdefault or defaultdict.
D) Use a set to count occurrences.

50. What is the output of sorting this array alphabetically by food name and then by number?


foods = ['Apple-1', 'Cucumber-5', 'Apple-33', 'Banana-17', 'Strawberry-15', 'Strawberry-4', 'Apple-4', 'Coconut-7', 'Banana-18', 'Strawberry-23']

A) ['Apple-1', 'Apple-4', 'Apple-33', 'Banana-17', 'Banana-18', 'Coconut-7', 'Cucumber-5', 'Strawberry-4', 'Strawberry-15', 'Strawberry-23']
B) ['Cucumber-5', 'Coconut-7', 'Banana-18', 'Banana-17', 'Apple-33', 'Apple-4', 'Apple-1', 'Strawberry-23', 'Strawberry-15', 'Strawberry-4']
C) ['Strawberry-4', 'Strawberry-15', 'Strawberry-23', 'Coconut-7', 'Cucumber-5', 'Banana-17', 'Banana-18', 'Apple-1', 'Apple-4', 'Apple-33']
D) ['Apple-1', 'Apple-33', 'Apple-4', 'Banana-17', 'Banana-18', 'Coconut-7', 'Cucumber-5', 'Strawberry-15', 'Strawberry-23', 'Strawberry-4']

51. How can you set up a database in Django?
A) Edit admin.py and run 'python manage.py createdb'
B) Edit settings.py and run 'python manage.py migrate'
C) Edit urls.py and run 'python manage.py makemigrations'
D) Edit models.py and run 'python manage.py syncdb'

52. What is the value of z after this code? z = 5 x = '456' while z > -1: y = x if y in ('1', '2', '3') or len(y) > 1: break x = x + y if y in ('4', '5', '6'): continue z = z - 1 print(z)
A) 10
B) 0
C) -1
D) 5

53. How do you insert an object at a given index in Python?
A) Use list[index] = object
B) Use list.add(index, object)
C) Use list.append()
D) Use list.insert(index, object)

54. What happens when you multiply a list containing a sublist and modify it?
A) Modifying a sublist will raise an error.
B) Each sublist is independent, so modifying one does not affect others.
C) All sublists are references to the same list, so modifying one affects all.
D) List multiplication creates deep copies of the sublist.

55. How can we make forms in Python?
A) Use SQLAlchemy for forms.
B) Use Django or Flask for web forms, Tkinter for GUI forms.
C) Use Flask for GUI forms, Tkinter for web forms.
D) Use PyQt for web forms, Django for GUI forms.

56. What is the output of this code?


lst1 = [1, 2, 3]
lst2 = lst1
lst3 = lst2
del lst2[1:2]
print(lst3)

A) []
B) [1, 2, 3]
C) [1, 3]
D) [2, 3]

57. How are arguments passed by value or by reference in Python?
A) By reference
B) By name
C) By object reference
D) By value

58. What is the purpose of *args and **kwargs in Python?
A) *args is for positional arguments, **kwargs for keyword arguments.
B) *args is for default arguments, **kwargs for variable arguments.
C) *args is for keyword arguments, **kwargs for positional arguments.
D) *args is for named arguments, **kwargs for unnamed arguments.

59. What is the difference between a list and a tuple in Python?
A) Lists are mutable, tuples are immutable.
B) Lists are immutable, tuples are mutable.
C) Both are immutable.
D) Both are mutable.

60. What is the purpose of Python decorators, and how can they be used?
A) Decorators are used to define classes.
B) Decorators are used to import functions and decorate them.
C) Decorators are used to extend function logic by handle redundant code among multiple functions by wrapping them.
D) Decorators are used to create anonymous functions.

61. What is the output of the following code and why?


class Foo:
    def __init__(self):
        self.x = 5
        self.bubbles = "Bubbles"
    def buzz(self):
        print(self.bubbles)
f = Foo()
f.buzz()

A) False, because a new list is created.
B) True, because the function returns the id of a new list.
C) True, because the list is modified in place.
D) False, because the ids are different.

62. What will be the output of `3 / 2` in Python 3?
A) 1.5
B) 2
C) Error
D) 1

63. How do you delete a file in Python?
A) Use os.delete('filename')
B) Use shutil.remove('filename')
C) Use file.delete('filename')
D) Use os.remove('filename')

64. How can you generate random numbers in Python?
A) Use the statistics module.
B) Use the numpy module.
C) Use the random module.
D) Use the math module.

65. Which pair best illustrates an abstract-concrete class relationship?
A) Truck - Window
B) Planet - Moon
C) Cat - Kitten
D) Colour - Red

66. How do you install Python on Windows and set the path variable?
A) Download from GitHub and manually set the path.
B) Download from python.org and check 'Add Python to PATH'.
C) Use pip to install Python.
D) Install via Anaconda.

67. What is the purpose of the property decorator in Python?
A) The property decorator is used to define static methods.
B) The property decorator allows methods to be accessed as attributes.
C) The property decorator is used to create class methods.
D) The property decorator is used to handle exceptions.

Friday, June 5, 2020

getattrib, Setattrib, hasattrib and delattrib in python

# `getattr(object, name[, default])` Function in Python

The `getattr(object, name[, default])` function returns the value of a named attribute of an object, where `name` must be a string. If the object has an attribute with the specified `name`, then the value of that attribute is returned. On the other hand, if the object does not have an attribute with `name`, then the value of `default` is returned, or `AttributeError` is raised if `default` is not provided.

```python

>>> t = ('This', 'is', 'a', 'tuple')

>>> t.index('is')

>>> getattr(t, 'index')

<built-in method index of tuple object at 0x10c15e680>

>>> getattr(t, 'index')('is')

```

when the attribute is not defined,

```python

>>> getattr(t, 'len')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'tuple' object has no attribute 'len'
>>> getattr(t, 'len', t.count)
<built-in method count of tuple object at 0x10c15e680>

```

Besides "normal" objects like tuples, lists, and class instances, getattr also accepts modules as arguments. Since modules are also objects in Python, the attributes of modules can be retrieved just like any attribute in an object.
>>> import uuid
>>> getattr(uuid, 'UUID')
<class 'uuid.UUID'>
>>> type(getattr(uuid, 'UUID'))
<type 'type'>
>>> isinstance(getattr(uuid, 'UUID'), type)
True
>>> callable(getattr(uuid, 'UUID'))
True
>>> getattr(uuid, 'UUID')('12345678123456781234567812345678')
UUID('12345678-1234-5678-1234-567812345678')

*****

# check for existence of an attribute
# hasattr #uses pythonic "Look Before You Leap" (LBYL) code style

>>> hasattr('abc', 'upper')
True
>>> hasattr('abc', 'lower')
True
>>> hasattr('abc', 'convert')
False

Using try-except way # "Easier to Ask for Forgiveness than Permission" (EAFP) code style

>>> try:
...     'abc'.upper()
... except AttributeError:
...     print("abc does not have attribute 'upper'")
...
'ABC'
>>> try:
...     'abc'.convert()
... except AttributeError:
...     print("abc does not have attribute 'convert'")
...
abc does not have attribute 'convert'

**********

hasattr vs dict

>>> class A(object):
...   foo = 1
...
>>> class B(A):
...   pass
...
>>> b = B()
>>> hasattr(b, 'foo')
True
>>> 'foo' in b.__dict__
False

#setattrib --Assigns a value to the object’s attribute given its name.

EX: setattr(x, ‘foobar’, 123) is equivalent to x.foobar = 123

Example 1

>>> class Foo:
...     def __init__(self, x):
...         self.x = x
...
>>> f = Foo(10)
>>> f.x
10
>>> setattr(f, 'x', 20)
>>> f.x
20
>>> setattr(f, 'y', 10)
>>> f.y
10
>>> f.y = 100
>>> f.y
100

you can dynamically add a function as a method to a class

>>> def b(self):
            print 'bar'

class Foo:
    pass
f = Foo()
print dir(f) #[‘__doc__’, ‘__module__’]
setattr(Foo, ‘bar’, b)
print dir(f) #[‘__doc__’, ‘__module__’, ‘bar’]
f.bar() #bar

you can setattr() on an instance of a class that
inherits from "object", but you can't on an instance of "object"
itself

>>> o = object()
>>> setattr(o, "x", 1000)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'object' object has no attribute 'x'

>>> class Object(object):pass
...
>>> o = Object()
>>> setattr(o, "x", 1000)
>>> o.x
1000

If __setattr__() wants to assign to an instance attribute,
it should not simply execute self.name = value — this would
cause a recursive call to itself. Instead, it should insert
the value in the dictionary of instance attributes,
e.g., self.__dict__[name] = value.
For new-style classes, rather than accessing the instance dictionary,
it should call the base class method with the same name,
for example, object.__setattr__(self, name, value).

****for blog****
The __slots__ declaration takes a sequence of instance variables and
reserves just enough space in each instance to hold a value for each
variable. Space is saved because __dict__ is not created for each instance.

When there is '__slots__', there won't be '__dict__' and '__weakref__'

delattr will delete the attribute

Example:

class Box:
    pass

box = Box()

# Create a width attribute.
setattr(box, "width", 15)

# The attribute exists.
if hasattr(box, "width"):
    print(True)

# Delete the width attribute.
delattr(box, "width")

# Width no longer exists.
if not hasattr(box, "width"):
    print(False)

Python tips

python tips:

In [1]: os.listdir(os.getcwd())==os.listdir(os.curdir)

Out[1]: True

In [2]: os.listdir(os.getcwd())==os.listdir('.')

Out[2]: True

*******
WAP for print a character its position times in a word.

word=raw_input("Enter the word:")

for i in word:
    print i*word.index(i)

Output:

******
WAP for print a character its ASCII values times in a word.



word=raw_input("Enter the word:")

for i in word:
 print i*ord(i)

#alternatively
# print [i*ord(i) for i in word]

Sunday, December 17, 2017

Initiating static local HTTP Server using different scripting languages

Initiating HTTP Server using different scripting languages

Python

python 2.x - python -m SimpleHTTPServer 8000

python 3.x - python -m http.server 8000

http-server (Node.js)

npm install -g http-server # install dependency

http-server -p 8000

node-static (Node.js)

npm install -g node-static # install dependency

static -p 8000

Ruby

ruby -rwebrick -e’WEBrick::HTTPServer.new(:Port => 3000, :DocumentRoot => Dir.pwd).start’

Ruby 1.9.2+

- ruby -run -ehttpd . -p3000

adsf (Ruby)

- gem install adsf # install dependency

- adsf -p 8000

Perl

----

cpan HTTP::Server::Brick # install dependency

- $ perl -MHTTP::Server::Brick -e ‘$s=HTTP::Server::Brick->new(port=>8000); $s->mount(“/”=>{path=>”.”}); $s->start’

Plack (Perl)

- cpan Plack # install dependency

- plackup -MPlack::App::Directory -e ‘Plack::App::Directory->new(root=>”.”);’ -p 8000

PHP (>= 5.4)

------------

php -S 127.0.0.1:8000

busybox httpd

-------------

busybox httpd -f -p 8000

webfs

-----

webfsd -F -p 8000

IIS Express

-----------

C:\> “C:\Program Files (x86)\IIS Express\iisexpress.exe” /path:C:\MyWeb /port:8000

AWS CLI installation procedure in centos/ RHEL

1) Make sure you have the epel repository installed
sudo yum install -y epel-release

2) Install needed packages
sudo yum install -y python2-pip

3) Install awscli
sudo yum install -y awscli

4) Configure awscli
aws configure --profile <profile_name>

4) Configure awscli.
If you do not add --profile <profile_name>, it will use the default profile.

aws configure --profile <profile_name>

Using the --profile allows you to create profiles for other accounts using different keys.
You can also create profiles for different regions.

Saturday, March 22, 2025

Apache Airflow: A One-Stop Guide for Junior Developers

History and Evolution

What is Airflow?

Architecture with Mermaid Diagram

Executor Modes with Mermaid Diagrams

LocalExecutor

CeleryExecutor

Sample Python DAG Code

Saturday, March 15, 2025

Introduction

Backend Expertise

Frontend Expertise

Databases

Python Expertise

Caching and Scheduling

Public Cloud Experience

Server-Based Architectures

Storage Solutions

Databases

Serverless Architectures

Container-based Environments

CI/CD and Infrastructure as Code

Agile Methodologies

Wednesday, September 6, 2023

Python Interview Quiz

Friday, June 5, 2020

hasattr vs __dict__

Sunday, December 17, 2017

hasattr vs dict