Skip to main content

Posts

Showing posts from February, 2024

Generative AI to Summarize the Urdu Text

Generative AI is like a creative machine, using data to dream up new things! It can write poems, paint pictures, even compose music. Imagine: feeding it words and getting a story back, or showing it a sketch and having it design a building! It's still young, but its potential is mind-blowing. To generate the summary for the Urdu language, I have trained an LLM model. Here are the steps you can follow to train your own Generative AI model for summarization. Install required packages. pip install transformers [ torch ] datasets == 2 .14.5 evaluate rouge_score --quiet Import necessary libraries. from datasets import load_dataset from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, GenerationConfig, TrainingArguments, Trainer import torch import time import evaluate import pandas as pd import numpy as np Load Dataset huggingface_dataset_name = "mirfan899/usummary" dataset = load_dataset(huggingface_dataset_name) Load pretrained LLM model model_na...