Posts

Showing posts from March, 2023

Accessing Data from an AWS data lake using AWS Lake Formation – Part 1 - Data Filtering based access control

Image
A data lake is a repository to store your data. Like a database, a data lake is expected to have the data in an organized manner, provide tools for data processing and data access, and have well defined methods for authentication and authorization. And unlike a database, data lake is expected to hold structured, semi-structured or unstructured data, and are envisioned to hold this data forever.   It is these subtle differences that make data lakes fit more for analytic needs such as deriving patterns, detailed comparisons, build an exhaustive story etc. A database can also be used for analytic use cases, but they are not meant for storing large volumes of data indefinitely, or store unstructured data, and this in turn puts a limit on how detailed of a result it can provide. Whether you need a data lake is a question that depends on the volume of data in your organization. If your organization data is small; you do not have much history or you do not have a need to store large history;