Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
version: 2
updates:
- package-ecosystem: github-actions
directory: "/"
schedule:
interval: weekly
- package-ecosystem: bundler
directory: "/"
schedule:
interval: weekly
27 changes: 27 additions & 0 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: codeql

on:
push:
branches: [main]
pull_request:
schedule:
- cron: "0 0 * * 0" # weekly

permissions:
actions: read
contents: read
security-events: write

jobs:
analyze:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
language: [ruby]
steps:
- uses: actions/checkout@v7
- uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
- uses: github/codeql-action/analyze@v3
110 changes: 58 additions & 52 deletions .github/workflows/python-arabic.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,76 +6,82 @@
pull_request:

jobs:
lint:
runs-on: ubuntu-latest
continue-on-error: true # 36 pre-existing violations; see TODO.complete/08-ruff-rababa-python.md
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with:
python-version: "3.11"
- run: pip install ruff
- run: ruff check python/
- run: ruff format --check python/

infer:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ['3.6', '3.7', '3.8', '3.9']
python-version: ["3.9"]

steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with:
python-version: ${{ matrix.python-version }}
cache: pip
cache-dependency-path: python/arabic/requirements.txt

- uses: actions/cache@v2
with:
path: ${{ env.pythonLocation }}
key: ${{ env.pythonLocation }}-${{ hashFiles('python/arabic/setup.py') }}-${{ hashFiles('python/arabic/requirements.txt') }}
- name: Install requirements
working-directory: ./python/arabic
run: |
pip install --upgrade --upgrade-strategy eager -r requirements.txt -e .

- name: Install requirements
working-directory: ./python/arabic
run: |
pip install --upgrade --upgrade-strategy eager -r requirements.txt -e .
- name: Download PyTorch model
working-directory: ./python/arabic
run: |
curl -sSL https://github.com/secryst/rababa-models/releases/download/0.1/2000000-snapshot.pt \
-o log_dir/CA_MSA.base.cbhg/models/2000000-snapshot.pt

- name: Download PyTorch model
working-directory: ./python/arabic
run: |
curl -sSL https://github.com/secryst/rababa-models/releases/download/0.1/2000000-snapshot.pt \
-o log_dir/CA_MSA.base.cbhg/models/2000000-snapshot.pt

- name: Run diacriticization
working-directory: ./python/arabic
run: |
python diacritize.py --model_kind "cbhg" --config config/cbhg.yml --text 'قطر'
- name: Run diacriticization
working-directory: ./python/arabic
run: |
python diacritize.py --model_kind "cbhg" --config config/cbhg.yml --text 'قطر'

train:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ['3.6', '3.7', '3.8', '3.9']
python-version: ["3.9"]

steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- uses: actions/cache@v2
with:
path: ${{ env.pythonLocation }}
key: ${{ env.pythonLocation }}-${{ hashFiles('python/setup.py') }}-${{ hashFiles('python/requirements.txt') }}
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with:
python-version: ${{ matrix.python-version }}
cache: pip
cache-dependency-path: python/arabic/requirements.txt

- name: Install requirements
working-directory: ./python/arabic
run: |
pip install --upgrade --upgrade-strategy eager -r requirements.txt -e .
- name: Install requirements
working-directory: ./python/arabic
run: |
pip install --upgrade --upgrade-strategy eager -r requirements.txt -e .

- name: Prepare dataset
working-directory: ./python/arabic
run: |
mkdir -p data/CA_MSA
touch data/CA_MSA/{eval,train,test}.csv
cd data
curl -sSL https://github.com/interscript/rababa-tashkeela/archive/refs/tags/v1.0.zip -o tashkeela.zip
unzip tashkeela.zip
for d in `ls rababa-tashkeela-1.0/tashkeela_val/*`; do cat $d >> CA_MSA/eval.csv; done
for d in `ls rababa-tashkeela-1.0/tashkeela_train/*`; do cat $d >> CA_MSA/train.csv; done
for d in `ls rababa-tashkeela-1.0/tashkeela_test/*`; do cat $d >> CA_MSA/test.csv; done
- name: Prepare dataset
working-directory: ./python/arabic
run: |
mkdir -p data/CA_MSA
touch data/CA_MSA/{eval,train,test}.csv
cd data
curl -sSL https://github.com/interscript/rababa-tashkeela/archive/refs/tags/v1.0.zip -o tashkeela.zip
unzip tashkeela.zip
for d in `ls rababa-tashkeela-1.0/tashkeela_val/*`; do cat $d >> CA_MSA/eval.csv; done
for d in `ls rababa-tashkeela-1.0/tashkeela_train/*`; do cat $d >> CA_MSA/train.csv; done
for d in `ls rababa-tashkeela-1.0/tashkeela_test/*`; do cat $d >> CA_MSA/test.csv; done

- name: Try training (WIP)
working-directory: ./python/arabic
run: |
python train.py --model "cbhg" --config config/test_cbhg.yml
- name: Try training (WIP)
working-directory: ./python/arabic
run: |
python train.py --model "cbhg" --config config/test_cbhg.yml
20 changes: 8 additions & 12 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ jobs:
release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v7

- uses: actions/setup-ruby@v1
- uses: ruby/setup-ruby@v1
with:
ruby-version: '2.7'
architecture: 'x64'
ruby-version: '3.3'
bundler-cache: true

- run: bundle install --jobs 4 --retry 3

Expand All @@ -23,14 +23,10 @@ jobs:

- name: Publish to rubygems.org
env:
RUBYGEMS_API_KEY: ${{secrets.INTERSCRIPT_RUBYGEMS_API_KEY}}
RUBYGEMS_API_KEY: ${{ secrets.INTERSCRIPT_RUBYGEMS_API_KEY }}
run: |
gem install gem-release
touch ~/.gem/credentials
cat > ~/.gem/credentials << EOF
---
:rubygems_api_key: ${RUBYGEMS_API_KEY}
EOF
mkdir -p ~/.gem
printf -- "---\n:rubygems_api_key: %s\n" "$RUBYGEMS_API_KEY" > ~/.gem/credentials
chmod 0600 ~/.gem/credentials
git status
gem install gem-release
gem release
24 changes: 11 additions & 13 deletions .github/workflows/ruby.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,21 +11,19 @@ jobs:
strategy:
fail-fast: false
matrix:
ruby-version: ['2.6', '2.7', '3.0', '3.1', '3.2']
ruby-version: ["3.3", "3.4"]

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v7

- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: ${{ matrix.ruby-version }}
bundler-cache: true
- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: ${{ matrix.ruby-version }}
bundler-cache: true

- name: Run rake
run: |
bundle exec rake
- name: Run rake
run: bundle exec rake

- name: Run standardrb (use bundle exec standardrb --fix)
run: |
bundle exec standardrb
- name: Run standardrb
run: bundle exec standardrb
15 changes: 15 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,18 @@ Gemfile.lock

.eggs
__pycache__

# Python training artifacts — never commit
python/log_dir/
python/data/
python/models/
python/__pycache__/
**/__pycache__/
*.pyc
*.pyo
*.onnx
*.pt

# Editor
.idea/
.vscode/
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Changelog

All notable changes to this project are documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [Latest]

See GitHub releases for detailed release notes: https://github.com/interscript/rababa/releases
42 changes: 42 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Contributing

Thanks for your interest in contributing!

## Development setup

```bash
git clone <this repo>
cd <repo>
bundle install # Ruby projects
# or
npm ci # JS projects
```

## Workflow

1. Fork → branch from `main`
2. Make changes with tests
3. Run `bundle exec rspec` (Ruby) or `npm test` (JS) locally
4. Run `bundle exec standardrb` (Ruby) or `npm run lint` (JS)
5. Open a PR with a clear description

## Code style

- Ruby: enforced by [StandardRB](https://github.com/standardrb/standard)
- JavaScript: enforced by ESLint + Prettier
- Python: enforced by ruff

## Commit messages

Use [Conventional Commits](https://www.conventionalcommits.org/):

```
feat: add new transliteration system
fix: correct off-by-one in CALT lookup
chore: bump dependencies
docs: clarify README
```

## Releases

Maintainers tag releases following semver. CI publishes on tag push.
1 change: 1 addition & 0 deletions README.adoc
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
image:https://github.com/interscript/rababa/actions/workflows/ruby.yml/badge.svg["CI status", link="https://github.com/interscript/rababa/actions/workflows/ruby.yml"]
= رُبابَة RABABA the Middle-Eastern Language Diacritization Library

Middle-Eastern Language diacritization is useful for several practical business
Expand Down
20 changes: 20 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Security Policy

## Supported Versions

The latest released version of this project receives security fixes.

## Reporting a Vulnerability

Please **do not** open public GitHub issues for security vulnerabilities.

Report privately via one of:

- **GitHub Security Advisories** — Security tab → "Report a vulnerability" (preferred)
- **Email** — [email protected]

We acknowledge reports within 72 hours and aim to ship a fix within 30 days for critical issues. Coordinated disclosure is supported.

## Disclosure

Public disclosure happens after a fix is released, on a timeline agreed with the reporter.
Loading
Loading